Jul 22 18:48:53.818014 ip-10-0-142-177 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 22 18:48:53.818021 ip-10-0-142-177 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 22 18:48:53.818030 ip-10-0-142-177 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 22 18:48:53.818554 ip-10-0-142-177 systemd[1]: Failed to start Kubernetes Kubelet. Jul 22 18:49:04.934135 ip-10-0-142-177 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 22 18:49:04.934150 ip-10-0-142-177 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3dcfb30641db4b46b748a28677be204a -- Jul 22 18:51:06.225810 ip-10-0-142-177 systemd[1]: Starting Kubernetes Kubelet... Jul 22 18:51:06.640230 ip-10-0-142-177 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 18:51:06.640230 ip-10-0-142-177 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 18:51:06.640230 ip-10-0-142-177 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 18:51:06.640230 ip-10-0-142-177 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 18:51:06.640230 ip-10-0-142-177 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 18:51:06.640919 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.640228 2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 18:51:06.650021 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.649993 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:51:06.650021 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650011 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:51:06.650021 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650015 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:51:06.650021 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650019 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:51:06.650021 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650022 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:51:06.650021 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650025 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:51:06.650021 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650030 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650034 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650038 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650041 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650044 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650046 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650049 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650052 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650055 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650058 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650060 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650063 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650066 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650068 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650071 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650074 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650076 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650079 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650082 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650084 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:51:06.650271 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650094 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650097 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650100 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650102 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650104 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650107 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650109 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650112 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650114 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650117 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650119 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650122 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650124 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650127 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650130 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650133 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650137 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650139 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650142 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650144 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:51:06.650767 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650147 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650150 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650153 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650157 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650161 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650164 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650167 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650169 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650171 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650174 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650176 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650179 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650182 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650184 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650187 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650190 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650193 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650195 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650198 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:51:06.651257 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650200 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650203 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650205 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650208 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650210 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650213 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650215 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650219 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650222 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650224 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650227 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650229 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650232 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650234 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650237 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650240 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650242 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650245 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650249 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:51:06.651754 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650253 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650256 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650261 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650264 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650266 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650269 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650272 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650275 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650278 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650281 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650283 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650286 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650289 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650291 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.650294 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651088 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651095 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651098 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651101 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:51:06.652211 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651104 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651108 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651111 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651113 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651116 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651119 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651121 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651125 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651127 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651130 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651133 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651135 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651139 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651144 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651147 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651150 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651153 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651156 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651158 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:51:06.652684 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651161 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651164 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651167 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651169 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651172 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651174 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651177 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651179 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651182 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651184 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651187 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651189 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651192 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651194 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651200 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651203 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651205 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651208 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651211 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651213 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:51:06.653159 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651215 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651218 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651221 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651223 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651225 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651228 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651231 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651238 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651242 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651247 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651251 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651254 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651258 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651261 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651264 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651266 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651269 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651272 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651275 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:51:06.653677 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651277 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651280 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651282 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651285 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651287 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651290 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651293 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651297 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651300 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651302 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651305 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651308 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651310 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651313 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651315 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651318 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651320 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651323 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651326 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651328 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:51:06.654147 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651331 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651334 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651336 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651339 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651342 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651345 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651347 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651350 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651352 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651355 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651358 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651360 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651363 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651365 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651367 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651370 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651372 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651862 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651868 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651874 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:51:06.654651 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651877 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651880 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651883 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651886 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651889 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651893 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651896 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651899 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651901 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651904 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651907 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651909 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651912 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651920 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651923 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651926 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651928 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651931 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651933 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651936 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:51:06.655118 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651938 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651941 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651943 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651946 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651948 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651950 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651953 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651955 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651958 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651962 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651965 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651969 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651973 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651975 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651978 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651981 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651983 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651986 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651988 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651991 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:51:06.655608 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651993 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651996 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.651999 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652001 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652003 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652006 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652014 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652017 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652020 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652023 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652025 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652028 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652030 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652034 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652037 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652040 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652043 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652046 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652049 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:51:06.656089 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652052 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652054 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652057 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652060 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652064 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652066 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652069 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652071 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652074 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652076 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652079 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652081 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652084 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652086 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652089 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652091 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652094 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652096 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652104 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:51:06.656593 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652107 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652116 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652118 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652121 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652123 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652126 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652129 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652131 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652134 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652136 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652139 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652141 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652143 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652146 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652149 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652151 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652154 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652157 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652225 2584 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652236 2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652247 2584 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 18:51:06.657054 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652252 2584 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652256 2584 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652260 2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652264 2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652268 2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652271 2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652274 2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652278 2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652281 2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652284 2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652287 2584 flags.go:64] FLAG: --cgroup-root="" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652290 2584 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652293 2584 flags.go:64] FLAG: --client-ca-file="" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652311 2584 flags.go:64] FLAG: --cloud-provider="external" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652316 2584 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652328 2584 flags.go:64] FLAG: --cluster-domain="" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652331 2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652334 2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652337 2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652340 2584 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652344 2584 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652347 2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652350 2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 18:51:06.657562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652353 2584 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652356 2584 flags.go:64] FLAG: --contention-profiling="false" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652359 2584 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652362 2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652365 2584 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652370 2584 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652375 2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652378 2584 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652380 2584 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652383 2584 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652386 2584 flags.go:64] FLAG: --enable-server="true" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652389 2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652396 2584 flags.go:64] FLAG: --event-burst="100" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652399 2584 flags.go:64] FLAG: --event-qps="50" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652402 2584 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652405 2584 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652408 2584 flags.go:64] FLAG: --eviction-hard="" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652411 2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652414 2584 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652417 2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652420 2584 flags.go:64] FLAG: --eviction-soft="" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652423 2584 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652426 2584 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652430 2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652434 2584 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 18:51:06.658150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652436 2584 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652439 2584 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652442 2584 flags.go:64] FLAG: --feature-gates="" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652445 2584 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652449 2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652452 2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652455 2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652458 2584 flags.go:64] FLAG: --healthz-port="10248" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652461 2584 flags.go:64] FLAG: --help="false" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652464 2584 flags.go:64] FLAG: --hostname-override="ip-10-0-142-177.ec2.internal" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652467 2584 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652470 2584 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652474 2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652477 2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652481 2584 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652484 2584 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652487 2584 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652490 2584 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652493 2584 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652496 2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652499 2584 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652502 2584 flags.go:64] FLAG: --kube-reserved="" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652505 2584 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652507 2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 18:51:06.658820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652510 2584 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652527 2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652532 2584 flags.go:64] FLAG: --lock-file="" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652536 2584 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652540 2584 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652544 2584 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652549 2584 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652553 2584 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652556 2584 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652559 2584 flags.go:64] FLAG: --logging-format="text" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652562 2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652565 2584 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652568 2584 flags.go:64] FLAG: --manifest-url="" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652571 2584 flags.go:64] FLAG: --manifest-url-header="" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652575 2584 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652578 2584 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652582 2584 flags.go:64] FLAG: --max-pods="110" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652585 2584 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652588 2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652590 2584 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652595 2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652598 2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652601 2584 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652604 2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652611 2584 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 18:51:06.659409 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652614 2584 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652617 2584 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652620 2584 flags.go:64] FLAG: --pod-cidr="" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652623 2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652629 2584 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652632 2584 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652635 2584 flags.go:64] FLAG: --pods-per-core="0" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652638 2584 flags.go:64] FLAG: --port="10250" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652641 2584 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652644 2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03edf96858d681636" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652647 2584 flags.go:64] FLAG: --qos-reserved="" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652650 2584 flags.go:64] FLAG: --read-only-port="10255" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652653 2584 flags.go:64] FLAG: --register-node="true" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652656 2584 flags.go:64] FLAG: --register-with-taints="" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652660 2584 flags.go:64] FLAG: --registry-burst="10" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652663 2584 flags.go:64] FLAG: --registry-qps="5" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652666 2584 flags.go:64] FLAG: --reserved-cpus="" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652669 2584 flags.go:64] FLAG: --reserved-memory="" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652673 2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652676 2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652679 2584 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652682 2584 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652684 2584 flags.go:64] FLAG: --runonce="false" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652687 2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652691 2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 18:51:06.660073 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652694 2584 flags.go:64] FLAG: --seccomp-default="false" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652697 2584 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652702 2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652705 2584 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652708 2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652711 2584 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652713 2584 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652716 2584 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652719 2584 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652722 2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652725 2584 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652728 2584 flags.go:64] FLAG: --system-cgroups="" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652731 2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652736 2584 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652739 2584 flags.go:64] FLAG: --tls-cert-file="" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652741 2584 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652748 2584 flags.go:64] FLAG: --tls-min-version="" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652751 2584 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652754 2584 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652757 2584 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652759 2584 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652762 2584 flags.go:64] FLAG: --v="2" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652766 2584 flags.go:64] FLAG: --version="false" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652771 2584 flags.go:64] FLAG: --vmodule="" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652774 2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 18:51:06.660706 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.652778 2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652928 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652932 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652935 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652938 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652941 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652943 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652946 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652948 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652952 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652956 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652959 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652961 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652964 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652967 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652969 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652972 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652975 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652977 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652980 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:51:06.661314 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652982 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652985 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652988 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652990 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652993 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652995 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.652998 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653000 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653003 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653006 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653008 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653011 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653014 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653017 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653019 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653021 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653024 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653026 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653029 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653032 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:51:06.661831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653034 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653038 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653042 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653044 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653047 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653049 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653052 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653056 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653060 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653063 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653066 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653069 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653072 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653075 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653078 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653082 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653085 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653088 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653091 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:51:06.662318 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653093 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653096 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653098 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653101 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653104 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653107 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653109 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653112 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653115 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653117 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653120 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653122 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653125 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653127 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653131 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653136 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653138 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653141 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653143 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653146 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:51:06.662808 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653148 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653151 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653154 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653157 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653159 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653162 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653165 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653168 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653170 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653173 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653176 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653178 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653181 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653183 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653186 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653189 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653193 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653196 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653199 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:51:06.663290 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653202 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.653205 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.653210 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.659699 2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.659719 2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.659737 2584 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.659743 2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659797 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659801 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659805 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659808 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659811 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659814 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659817 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:51:06.663837 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659819 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659822 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659824 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659827 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659829 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659832 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659834 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659837 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659840 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659843 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659845 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659848 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659850 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659854 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659858 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659862 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659865 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659869 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659872 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:51:06.664196 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659875 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659877 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659880 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659883 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659886 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659888 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659891 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659894 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659897 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659900 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659902 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659906 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659909 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659912 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659915 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659918 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659920 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659923 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659925 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659928 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:51:06.664688 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659930 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659932 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659935 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659937 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659940 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659943 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659945 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659948 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659950 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659953 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659956 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659959 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659962 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659964 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659967 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659970 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659972 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659975 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659977 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659979 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:51:06.665184 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659982 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659984 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659987 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659989 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659991 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659994 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659996 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.659998 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660001 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660003 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660005 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660008 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660010 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660013 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660015 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660017 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660020 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660022 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660024 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:51:06.665831 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660027 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660031 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660035 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660038 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660041 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660044 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660047 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660049 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660052 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660055 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660057 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660059 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660062 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660065 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.660070 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 18:51:06.666288 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660189 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660194 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660197 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660200 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660203 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660205 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660208 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660210 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660213 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660216 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660218 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660221 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660223 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660225 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660228 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660230 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660233 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660236 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660239 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660241 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:51:06.666671 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660243 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660246 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660248 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660251 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660254 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660256 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660259 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660261 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660264 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660266 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660269 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660271 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660273 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660276 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660278 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660282 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660285 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660288 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660291 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:51:06.667164 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660294 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660296 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660299 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660301 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660304 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660307 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660309 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660311 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660314 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660316 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660319 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660321 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660324 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660326 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660329 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660331 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660333 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660336 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660339 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660341 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:51:06.667658 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660344 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660347 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660349 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660351 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660354 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660356 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660358 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660361 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660363 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660366 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660368 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660371 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660374 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660376 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660380 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660383 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660386 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660388 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660390 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:51:06.668149 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660393 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660395 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660398 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660400 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660403 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660405 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660408 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660410 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660412 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660415 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660417 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660420 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660423 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660425 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660427 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660430 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660432 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660434 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660437 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660440 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:51:06.668660 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:06.660443 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:51:06.669198 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.660448 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 18:51:06.669198 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.660554 2584 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 18:51:06.669198 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.662389 2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 18:51:06.669198 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.664835 2584 server.go:1019] "Starting client certificate rotation" Jul 22 18:51:06.669198 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.664938 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 18:51:06.669198 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.664973 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 18:51:06.669198 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.667195 2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 18:51:06.669198 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.668258 2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 18:51:06.680002 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.679985 2584 log.go:25] "Validated CRI v1 runtime API" Jul 22 18:51:06.681899 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.681883 2584 log.go:25] "Validated CRI v1 image API" Jul 22 18:51:06.684866 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.684851 2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 18:51:06.687744 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.687725 2584 fs.go:135] Filesystem UUIDs: map[60f2a872-fc98-41bd-9fa6-c90ff984553c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7c35695f-266c-4ea4-9e64-e7be4447d6df:/dev/nvme0n1p3] Jul 22 18:51:06.687810 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.687743 2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 18:51:06.693585 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.693460 2584 manager.go:216] Machine: {Timestamp:2026-07-22 18:51:06.691364313 +0000 UTC m=+0.353938643 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100606 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec287b47417ecc6f6d238d6f81a406d4 SystemUUID:ec287b47-417e-cc6f-6d23-8d6f81a406d4 BootID:3dcfb306-41db-4b46-b748-a28677be204a Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2c:1d:04:2b:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2c:1d:04:2b:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:63:ca:dc:cd:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 18:51:06.693585 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.693579 2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 18:51:06.693693 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.693651 2584 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 18:51:06.694556 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.694533 2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 18:51:06.694687 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.694559 2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-177.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 18:51:06.695881 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.695867 2584 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 18:51:06.695920 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.695888 2584 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 18:51:06.695920 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.695901 2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 18:51:06.696616 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.696605 2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 18:51:06.697816 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.697806 2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 18:51:06.700691 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.700679 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 22 18:51:06.700811 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.700801 2584 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 18:51:06.700851 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.700830 2584 kubelet.go:486] "Attempting to sync node with API server" Jul 22 18:51:06.700851 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.700839 2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 18:51:06.700851 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.700849 2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 18:51:06.700974 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.700857 2584 kubelet.go:398] "Adding apiserver pod source" Jul 22 18:51:06.700974 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.700866 2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 18:51:06.701293 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.701254 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 18:51:06.701347 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.701308 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 18:51:06.701702 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.701684 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 18:51:06.703367 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.703351 2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 18:51:06.705029 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.705009 2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 18:51:06.705102 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.705044 2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 18:51:06.706851 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706839 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 18:51:06.706899 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706856 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 18:51:06.706899 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706863 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 18:51:06.706899 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706868 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 18:51:06.706899 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706879 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 18:51:06.706899 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706888 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 18:51:06.706899 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706900 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 18:51:06.707067 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706907 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 18:51:06.707067 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706914 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 18:51:06.707067 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706920 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 18:51:06.707067 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706936 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 18:51:06.707067 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.706941 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 18:51:06.707898 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.707887 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 18:51:06.707898 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.707897 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 18:51:06.709198 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.709172 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-177.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 18:51:06.709306 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.709211 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 18:51:06.711029 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.711010 2584 server.go:1268] "Started kubelet" Jul 22 18:51:06.711145 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.711093 2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 18:51:06.711241 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.711161 2584 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 18:51:06.711241 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.711154 2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 18:51:06.711755 ip-10-0-142-177 systemd[1]: Started Kubernetes Kubelet. Jul 22 18:51:06.712965 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.712949 2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 18:51:06.713291 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.713278 2584 server.go:310] "Adding debug handlers to kubelet server" Jul 22 18:51:06.716809 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.716790 2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 18:51:06.718471 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.718452 2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 18:51:06.718725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.718702 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 18:51:06.718853 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.718809 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:06.719395 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.719379 2584 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 18:51:06.719575 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.719563 2584 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 18:51:06.719714 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.719695 2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 18:51:06.720063 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.720051 2584 reconstruct.go:97] "Volume reconstruction finished" Jul 22 18:51:06.720180 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.720160 2584 reconciler.go:29] "Reconciler: start to sync state" Jul 22 18:51:06.720273 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.720109 2584 factory.go:55] Registering systemd factory Jul 22 18:51:06.720273 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.720201 2584 factory.go:223] Registration of the systemd container factory successfully Jul 22 18:51:06.720803 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.720784 2584 factory.go:153] Registering CRI-O factory Jul 22 18:51:06.720888 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.720829 2584 factory.go:223] Registration of the crio container factory successfully Jul 22 18:51:06.720934 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.720900 2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 18:51:06.720991 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.720924 2584 factory.go:103] Registering Raw factory Jul 22 18:51:06.721085 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.721071 2584 manager.go:1196] Started watching for new ooms in manager Jul 22 18:51:06.721940 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.721912 2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-177.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 18:51:06.722136 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.722121 2584 manager.go:349] Starting recovery of all containers Jul 22 18:51:06.724557 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.721403 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-177.ec2.internal.18c4b146098340a0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-177.ec2.internal,UID:ip-10-0-142-177.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-177.ec2.internal,},FirstTimestamp:2026-07-22 18:51:06.710982816 +0000 UTC m=+0.373557146,LastTimestamp:2026-07-22 18:51:06.710982816 +0000 UTC m=+0.373557146,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-177.ec2.internal,}" Jul 22 18:51:06.724643 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.723380 2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-177.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 18:51:06.726123 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.726101 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wr4qv" Jul 22 18:51:06.731221 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.731194 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 18:51:06.734505 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.734380 2584 manager.go:354] Recovery completed Jul 22 18:51:06.734823 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.734802 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wr4qv" Jul 22 18:51:06.736157 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.736141 2584 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 22 18:51:06.739069 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.739055 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:51:06.740817 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.740801 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:51:06.740870 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.740832 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:51:06.740870 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.740846 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:51:06.741377 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.741364 2584 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 18:51:06.741377 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.741372 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:51:06.741377 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.741376 2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 18:51:06.741489 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.741390 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 22 18:51:06.742063 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.742051 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:51:06.742112 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.742074 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:51:06.742112 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.742084 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:51:06.742411 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.742401 2584 policy_none.go:49] "None policy: Start" Jul 22 18:51:06.742442 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.742415 2584 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 18:51:06.742442 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.742425 2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 18:51:06.743029 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.742965 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-177.ec2.internal.18c4b1460b4a7dd9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-177.ec2.internal,UID:ip-10-0-142-177.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-177.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-177.ec2.internal,},FirstTimestamp:2026-07-22 18:51:06.740817369 +0000 UTC m=+0.403391699,LastTimestamp:2026-07-22 18:51:06.740817369 +0000 UTC m=+0.403391699,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-177.ec2.internal,}" Jul 22 18:51:06.743465 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.743454 2584 policy_none.go:47] "Start" Jul 22 18:51:06.784075 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.784061 2584 manager.go:337] "Starting Device Plugin manager" Jul 22 18:51:06.784159 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.784096 2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 18:51:06.784159 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.784107 2584 server.go:87] "Starting device plugin registration server" Jul 22 18:51:06.784338 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.784328 2584 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 18:51:06.784374 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.784342 2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 18:51:06.784705 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.784532 2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 18:51:06.784705 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.784623 2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 18:51:06.784705 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.784632 2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 18:51:06.785121 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.785102 2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 18:51:06.785182 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.785155 2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:06.884738 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.884710 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:51:06.885897 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.885812 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:51:06.885897 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.885854 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:51:06.885897 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.885869 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:51:06.886052 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.885907 2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-177.ec2.internal" Jul 22 18:51:06.894219 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.894178 2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-177.ec2.internal" Jul 22 18:51:06.894219 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.894199 2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-177.ec2.internal\": node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:06.898626 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.898590 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 18:51:06.899980 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.899961 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 18:51:06.900059 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.899997 2584 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 18:51:06.900059 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.900020 2584 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 18:51:06.900124 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.900060 2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 18:51:06.902079 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:06.902062 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:51:06.910097 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:06.908825 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:07.000764 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.000722 2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal"] Jul 22 18:51:07.000853 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.000821 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:51:07.001636 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.001622 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:51:07.001702 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.001648 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:51:07.001702 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.001658 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:51:07.003343 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.003331 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:51:07.003477 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.003460 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.003541 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.003497 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:51:07.004054 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.004032 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:51:07.004054 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.004047 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:51:07.004197 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.004060 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:51:07.004197 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.004071 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:51:07.004197 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.004074 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:51:07.004197 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.004081 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:51:07.005383 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.005367 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.005467 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.005390 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:51:07.006045 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.006029 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:51:07.006112 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.006062 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:51:07.006112 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.006079 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:51:07.009159 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:07.009141 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:07.022529 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.022495 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2710698bde29e793f9633bc0d191b7e7-config\") pod \"kube-apiserver-proxy-ip-10-0-142-177.ec2.internal\" (UID: \"2710698bde29e793f9633bc0d191b7e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.022601 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.022539 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6074a94eb729fb72ee8c105664c9c17b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal\" (UID: \"6074a94eb729fb72ee8c105664c9c17b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.022601 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.022557 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6074a94eb729fb72ee8c105664c9c17b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal\" (UID: \"6074a94eb729fb72ee8c105664c9c17b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.028056 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:07.028039 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-177.ec2.internal\" not found" node="ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.031341 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:07.031324 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-177.ec2.internal\" not found" node="ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.109836 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:07.109807 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:07.123206 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.123182 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6074a94eb729fb72ee8c105664c9c17b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal\" (UID: \"6074a94eb729fb72ee8c105664c9c17b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.123323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.123211 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2710698bde29e793f9633bc0d191b7e7-config\") pod \"kube-apiserver-proxy-ip-10-0-142-177.ec2.internal\" (UID: \"2710698bde29e793f9633bc0d191b7e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.123323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.123230 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6074a94eb729fb72ee8c105664c9c17b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal\" (UID: \"6074a94eb729fb72ee8c105664c9c17b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.123323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.123291 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6074a94eb729fb72ee8c105664c9c17b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal\" (UID: \"6074a94eb729fb72ee8c105664c9c17b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.123323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.123301 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2710698bde29e793f9633bc0d191b7e7-config\") pod \"kube-apiserver-proxy-ip-10-0-142-177.ec2.internal\" (UID: \"2710698bde29e793f9633bc0d191b7e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.123323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.123312 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6074a94eb729fb72ee8c105664c9c17b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal\" (UID: \"6074a94eb729fb72ee8c105664c9c17b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.210694 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:07.210614 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:07.311262 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:07.311241 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:07.331110 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.331076 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.334169 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.334146 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.414717 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:07.414680 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:07.515705 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:07.515624 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:07.621332 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:07.621283 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:07.666390 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.666364 2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 18:51:07.666928 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.666525 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 18:51:07.721530 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:07.721495 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-177.ec2.internal\" not found" Jul 22 18:51:07.731750 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.731725 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:51:07.738201 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.738172 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 18:46:06 +0000 UTC" deadline="2028-01-19 00:37:10.545618648 +0000 UTC" Jul 22 18:51:07.738201 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.738200 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13085h46m2.807421944s" Jul 22 18:51:07.821603 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.821491 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.832433 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.832407 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 18:51:07.834833 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.834819 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" Jul 22 18:51:07.847745 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:07.847724 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 18:51:08.040642 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.040617 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:51:08.209210 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.209126 2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:51:08.644207 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.644120 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:51:08.704548 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.704510 2584 apiserver.go:52] "Watching apiserver" Jul 22 18:51:08.713123 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.713100 2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 18:51:08.713507 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.713479 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-vmpdl","openshift-image-registry/node-ca-g2xmq","openshift-multus/multus-additional-cni-plugins-bh8z5","openshift-multus/network-metrics-daemon-l98h2","openshift-ovn-kubernetes/ovnkube-node-xg4p5","kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs","openshift-cluster-node-tuning-operator/tuned-gb4gp","openshift-dns/node-resolver-lpxbp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal","openshift-multus/multus-867pp","openshift-network-diagnostics/network-check-target-f2dck","openshift-network-operator/iptables-alerter-hzw9m","kube-system/global-pull-secret-syncer-247b8"] Jul 22 18:51:08.715078 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.715060 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.716353 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.716333 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:08.717646 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.717621 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.718115 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.718098 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 18:51:08.718115 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.718108 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 18:51:08.718232 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.718114 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 18:51:08.718232 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.718123 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jq4jg\"" Jul 22 18:51:08.718995 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.718973 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:08.719411 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.719391 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 18:51:08.719411 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.719397 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:51:08.719411 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.719397 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:08.719686 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.719515 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bzgp6\"" Jul 22 18:51:08.719748 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.719516 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 18:51:08.721679 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.720206 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 18:51:08.721679 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.720530 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 18:51:08.721679 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.720927 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vqbvg\"" Jul 22 18:51:08.721679 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.721031 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 18:51:08.721679 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.721316 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 18:51:08.721679 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.721466 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 18:51:08.721679 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.721628 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 18:51:08.722061 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.721942 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.723617 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.723576 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:08.723710 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.723652 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:08.724891 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.724868 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 18:51:08.724990 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.724870 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 18:51:08.725040 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.725007 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:08.725314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.725294 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7bjfx\"" Jul 22 18:51:08.725416 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.725333 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 18:51:08.725687 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.725673 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 18:51:08.725761 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.725704 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 18:51:08.725806 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.725760 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 18:51:08.726310 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.726294 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:08.727571 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.727553 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:08.728296 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.728277 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 18:51:08.728392 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.728332 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 18:51:08.728392 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.728339 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nrq5q\"" Jul 22 18:51:08.728748 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.728729 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cgln8\"" Jul 22 18:51:08.728822 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.728806 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 18:51:08.728901 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.728884 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.729123 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.729101 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 18:51:08.730263 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.730246 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-867pp" Jul 22 18:51:08.730997 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.730435 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tcggk\"" Jul 22 18:51:08.730997 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.730444 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 18:51:08.730997 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.730630 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 18:51:08.730997 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.730728 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 18:51:08.731665 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.731645 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:51:08.731754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.731664 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hjhpf\"" Jul 22 18:51:08.731754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.731674 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 18:51:08.731754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.731658 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:08.731882 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.731770 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:08.732419 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732402 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ad39009-35e7-48ce-bb70-a0b607148a0d-host-slash\") pod \"iptables-alerter-hzw9m\" (UID: \"0ad39009-35e7-48ce-bb70-a0b607148a0d\") " pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:08.732457 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732428 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.732505 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732451 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:08.732505 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732477 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-run-systemd\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.732505 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732499 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-var-lib-openvswitch\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.732680 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732547 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-sys-fs\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.732680 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732580 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.732680 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732596 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-slash\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.732680 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732610 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-run-ovn\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.732680 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732624 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4575c424-46ed-4880-8fec-5198c2a66f3b-ovnkube-script-lib\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.732680 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732661 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-system-cni-dir\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.732680 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732676 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.732906 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732691 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.732906 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732720 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.732906 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732749 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-device-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.732906 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732781 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-cni-binary-copy\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.732906 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732808 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-etc-openvswitch\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.732906 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732836 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-run-openvswitch\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.732906 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732862 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 18:51:08.732906 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732862 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.732906 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732890 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5mz6j\"" Jul 22 18:51:08.732906 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732910 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn2p6\" (UniqueName: \"kubernetes.io/projected/4575c424-46ed-4880-8fec-5198c2a66f3b-kube-api-access-zn2p6\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732929 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-log-socket\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732943 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4575c424-46ed-4880-8fec-5198c2a66f3b-ovnkube-config\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732958 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbcwc\" (UniqueName: \"kubernetes.io/projected/0ad39009-35e7-48ce-bb70-a0b607148a0d-kube-api-access-pbcwc\") pod \"iptables-alerter-hzw9m\" (UID: \"0ad39009-35e7-48ce-bb70-a0b607148a0d\") " pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.732973 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-os-release\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733002 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljftd\" (UniqueName: \"kubernetes.io/projected/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-kube-api-access-ljftd\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733050 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-kubelet\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733085 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-systemd-units\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733112 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-cni-bin\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733145 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-cni-netd\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733166 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scgsv\" (UniqueName: \"kubernetes.io/projected/30190c62-e518-4aad-9d3b-2d75330a372b-kube-api-access-scgsv\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733196 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ad39009-35e7-48ce-bb70-a0b607148a0d-iptables-alerter-script\") pod \"iptables-alerter-hzw9m\" (UID: \"0ad39009-35e7-48ce-bb70-a0b607148a0d\") " pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733233 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-cnibin\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733250 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jqlk\" (UniqueName: \"kubernetes.io/projected/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-kube-api-access-2jqlk\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733264 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-run-netns\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733284 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4575c424-46ed-4880-8fec-5198c2a66f3b-ovn-node-metrics-cert\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.733314 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733300 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-socket-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.733959 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733322 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-registration-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.733959 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733339 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-etc-selinux\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.733959 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733361 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-node-log\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.733959 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.733400 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4575c424-46ed-4880-8fec-5198c2a66f3b-env-overrides\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.736127 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.736108 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 18:51:08.759537 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.759491 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-q98mk" Jul 22 18:51:08.767584 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.767566 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q98mk" Jul 22 18:51:08.821366 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.821340 2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 18:51:08.833654 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833625 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-slash\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.833774 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833659 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4575c424-46ed-4880-8fec-5198c2a66f3b-ovnkube-script-lib\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.833774 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833689 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk42j\" (UniqueName: \"kubernetes.io/projected/5c280d2a-6833-4775-a794-317b88a86303-kube-api-access-jk42j\") pod \"node-ca-g2xmq\" (UID: \"5c280d2a-6833-4775-a794-317b88a86303\") " pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:08.833774 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833705 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-slash\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.833774 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833713 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-sysconfig\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.833774 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833754 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-run\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.834002 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833801 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61b89140-235e-4032-ad7f-f18c4a976dcf-tmp\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.834002 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833845 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-system-cni-dir\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.834002 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833874 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.834002 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833900 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-cnibin\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.834002 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833926 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-run-k8s-cni-cncf-io\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.834002 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833951 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/04d29c18-d8ce-4440-8438-3e217fb921df-konnectivity-ca\") pod \"konnectivity-agent-vmpdl\" (UID: \"04d29c18-d8ce-4440-8438-3e217fb921df\") " pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:08.834002 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.833953 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-system-cni-dir\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834012 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834053 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-etc-openvswitch\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834080 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn2p6\" (UniqueName: \"kubernetes.io/projected/4575c424-46ed-4880-8fec-5198c2a66f3b-kube-api-access-zn2p6\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834110 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834113 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-multus-socket-dir-parent\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834120 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834125 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-etc-openvswitch\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834149 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-var-lib-cni-bin\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834174 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-systemd\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834199 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-log-socket\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834222 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-run-multus-certs\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834249 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5bf105fc-1d98-4507-9129-3a5d1027d705-dbus\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834270 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834289 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbcwc\" (UniqueName: \"kubernetes.io/projected/0ad39009-35e7-48ce-bb70-a0b607148a0d-kube-api-access-pbcwc\") pod \"iptables-alerter-hzw9m\" (UID: \"0ad39009-35e7-48ce-bb70-a0b607148a0d\") " pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:08.834306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834309 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-os-release\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834335 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-kubelet\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834267 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-log-socket\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834364 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-systemd-units\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834339 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4575c424-46ed-4880-8fec-5198c2a66f3b-ovnkube-script-lib\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834392 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-cni-bin\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834405 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-kubelet\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834409 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-os-release\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834420 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-os-release\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834456 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q72zj\" (UniqueName: \"kubernetes.io/projected/62f16011-c0a3-42ac-ace9-fa028fc57f76-kube-api-access-q72zj\") pod \"node-resolver-lpxbp\" (UID: \"62f16011-c0a3-42ac-ace9-fa028fc57f76\") " pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834484 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ad39009-35e7-48ce-bb70-a0b607148a0d-iptables-alerter-script\") pod \"iptables-alerter-hzw9m\" (UID: \"0ad39009-35e7-48ce-bb70-a0b607148a0d\") " pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834482 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-cni-bin\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834504 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jqlk\" (UniqueName: \"kubernetes.io/projected/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-kube-api-access-2jqlk\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834552 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4873f3b-f674-41ca-bc4a-44595135d951-cni-binary-copy\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834569 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-etc-kubernetes\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834516 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-systemd-units\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834622 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62f16011-c0a3-42ac-ace9-fa028fc57f76-hosts-file\") pod \"node-resolver-lpxbp\" (UID: \"62f16011-c0a3-42ac-ace9-fa028fc57f76\") " pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:08.835131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834648 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-node-log\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834665 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4575c424-46ed-4880-8fec-5198c2a66f3b-env-overrides\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834688 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-var-lib-cni-multus\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834703 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c280d2a-6833-4775-a794-317b88a86303-host\") pod \"node-ca-g2xmq\" (UID: \"5c280d2a-6833-4775-a794-317b88a86303\") " pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834766 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-node-log\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834807 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-sysctl-d\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834841 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-lib-modules\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834868 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc98d\" (UniqueName: \"kubernetes.io/projected/61b89140-235e-4032-ad7f-f18c4a976dcf-kube-api-access-vc98d\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834896 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ad39009-35e7-48ce-bb70-a0b607148a0d-host-slash\") pod \"iptables-alerter-hzw9m\" (UID: \"0ad39009-35e7-48ce-bb70-a0b607148a0d\") " pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834923 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834951 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-run-systemd\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834966 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ad39009-35e7-48ce-bb70-a0b607148a0d-host-slash\") pod \"iptables-alerter-hzw9m\" (UID: \"0ad39009-35e7-48ce-bb70-a0b607148a0d\") " pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.834980 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-var-lib-openvswitch\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835008 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-system-cni-dir\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835032 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c280d2a-6833-4775-a794-317b88a86303-serviceca\") pod \"node-ca-g2xmq\" (UID: \"5c280d2a-6833-4775-a794-317b88a86303\") " pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835056 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-var-lib-openvswitch\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835061 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0ad39009-35e7-48ce-bb70-a0b607148a0d-iptables-alerter-script\") pod \"iptables-alerter-hzw9m\" (UID: \"0ad39009-35e7-48ce-bb70-a0b607148a0d\") " pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:08.835945 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835058 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-sys-fs\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835074 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-run-systemd\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835103 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-run-ovn\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835111 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-sys-fs\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835132 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-hostroot\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835136 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4575c424-46ed-4880-8fec-5198c2a66f3b-env-overrides\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835143 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-run-ovn\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835158 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5bf105fc-1d98-4507-9129-3a5d1027d705-kubelet-config\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835195 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-kubernetes\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835221 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835251 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f16011-c0a3-42ac-ace9-fa028fc57f76-tmp-dir\") pod \"node-resolver-lpxbp\" (UID: \"62f16011-c0a3-42ac-ace9-fa028fc57f76\") " pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835289 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/04d29c18-d8ce-4440-8438-3e217fb921df-agent-certs\") pod \"konnectivity-agent-vmpdl\" (UID: \"04d29c18-d8ce-4440-8438-3e217fb921df\") " pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835299 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-run-ovn-kubernetes\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835324 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-device-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835350 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-cni-binary-copy\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835378 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-run-openvswitch\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835404 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.836927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835433 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-var-lib-kubelet\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835452 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-run-openvswitch\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835459 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-tuned\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835352 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-device-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835479 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835486 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnzzt\" (UniqueName: \"kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt\") pod \"network-check-target-f2dck\" (UID: \"34dba969-9e7c-44f9-838a-1f6c70c66aa7\") " pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835514 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4575c424-46ed-4880-8fec-5198c2a66f3b-ovnkube-config\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835531 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835559 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-multus-conf-dir\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835584 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-sys\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835622 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljftd\" (UniqueName: \"kubernetes.io/projected/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-kube-api-access-ljftd\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835842 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-cni-binary-copy\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835890 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-cni-netd\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.835990 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-multus-cni-dir\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836005 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-cni-netd\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836013 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-modprobe-d\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836058 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-host\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.837567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836088 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4575c424-46ed-4880-8fec-5198c2a66f3b-ovnkube-config\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836090 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scgsv\" (UniqueName: \"kubernetes.io/projected/30190c62-e518-4aad-9d3b-2d75330a372b-kube-api-access-scgsv\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836157 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-cnibin\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836181 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-run-netns\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836196 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4575c424-46ed-4880-8fec-5198c2a66f3b-ovn-node-metrics-cert\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836217 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4873f3b-f674-41ca-bc4a-44595135d951-multus-daemon-config\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836255 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-socket-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836268 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4575c424-46ed-4880-8fec-5198c2a66f3b-host-run-netns\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836314 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-cnibin\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836362 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-registration-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836393 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-registration-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836391 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-etc-selinux\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836435 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-run-netns\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836463 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-var-lib-kubelet\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836490 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836512 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-etc-selinux\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836521 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdpjr\" (UniqueName: \"kubernetes.io/projected/f4873f3b-f674-41ca-bc4a-44595135d951-kube-api-access-vdpjr\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.838185 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836546 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/30190c62-e518-4aad-9d3b-2d75330a372b-socket-dir\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.838703 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836575 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-sysctl-conf\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.838703 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.836581 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:08.838703 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836607 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.838703 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.836652 2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 18:51:08.838703 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.836709 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs podName:d6610a82-7a15-4b2f-9d80-8d2eae96e69e nodeName:}" failed. No retries permitted until 2026-07-22 18:51:09.336652039 +0000 UTC m=+2.999226381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs") pod "network-metrics-daemon-l98h2" (UID: "d6610a82-7a15-4b2f-9d80-8d2eae96e69e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:08.838703 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.837052 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.840031 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.840013 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4575c424-46ed-4880-8fec-5198c2a66f3b-ovn-node-metrics-cert\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.849650 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.849625 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn2p6\" (UniqueName: \"kubernetes.io/projected/4575c424-46ed-4880-8fec-5198c2a66f3b-kube-api-access-zn2p6\") pod \"ovnkube-node-xg4p5\" (UID: \"4575c424-46ed-4880-8fec-5198c2a66f3b\") " pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:08.850138 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.850119 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbcwc\" (UniqueName: \"kubernetes.io/projected/0ad39009-35e7-48ce-bb70-a0b607148a0d-kube-api-access-pbcwc\") pod \"iptables-alerter-hzw9m\" (UID: \"0ad39009-35e7-48ce-bb70-a0b607148a0d\") " pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:08.850323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.850300 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scgsv\" (UniqueName: \"kubernetes.io/projected/30190c62-e518-4aad-9d3b-2d75330a372b-kube-api-access-scgsv\") pod \"aws-ebs-csi-driver-node-tbmfs\" (UID: \"30190c62-e518-4aad-9d3b-2d75330a372b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:08.850389 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.850300 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jqlk\" (UniqueName: \"kubernetes.io/projected/124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac-kube-api-access-2jqlk\") pod \"multus-additional-cni-plugins-bh8z5\" (UID: \"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac\") " pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:08.850389 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.850346 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljftd\" (UniqueName: \"kubernetes.io/projected/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-kube-api-access-ljftd\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:08.937037 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.936931 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-multus-socket-dir-parent\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937037 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.936977 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-var-lib-cni-bin\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937037 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.936997 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-systemd\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.937037 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937014 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-run-multus-certs\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937050 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-run-multus-certs\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937048 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-multus-socket-dir-parent\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937080 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-systemd\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.937323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937080 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-var-lib-cni-bin\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937110 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5bf105fc-1d98-4507-9129-3a5d1027d705-dbus\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:08.937323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937145 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:08.937323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937197 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-os-release\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937215 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q72zj\" (UniqueName: \"kubernetes.io/projected/62f16011-c0a3-42ac-ace9-fa028fc57f76-kube-api-access-q72zj\") pod \"node-resolver-lpxbp\" (UID: \"62f16011-c0a3-42ac-ace9-fa028fc57f76\") " pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:08.937323 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937253 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5bf105fc-1d98-4507-9129-3a5d1027d705-dbus\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:08.937323 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.937276 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937327 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4873f3b-f674-41ca-bc4a-44595135d951-cni-binary-copy\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.937343 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret podName:5bf105fc-1d98-4507-9129-3a5d1027d705 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:09.437320998 +0000 UTC m=+3.099895337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret") pod "global-pull-secret-syncer-247b8" (UID: "5bf105fc-1d98-4507-9129-3a5d1027d705") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937372 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-etc-kubernetes\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937375 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-os-release\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937397 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62f16011-c0a3-42ac-ace9-fa028fc57f76-hosts-file\") pod \"node-resolver-lpxbp\" (UID: \"62f16011-c0a3-42ac-ace9-fa028fc57f76\") " pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937433 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-etc-kubernetes\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937448 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/62f16011-c0a3-42ac-ace9-fa028fc57f76-hosts-file\") pod \"node-resolver-lpxbp\" (UID: \"62f16011-c0a3-42ac-ace9-fa028fc57f76\") " pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937448 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-var-lib-cni-multus\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937478 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c280d2a-6833-4775-a794-317b88a86303-host\") pod \"node-ca-g2xmq\" (UID: \"5c280d2a-6833-4775-a794-317b88a86303\") " pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937493 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-sysctl-d\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937509 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-lib-modules\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937525 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c280d2a-6833-4775-a794-317b88a86303-host\") pod \"node-ca-g2xmq\" (UID: \"5c280d2a-6833-4775-a794-317b88a86303\") " pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937491 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-var-lib-cni-multus\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937552 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc98d\" (UniqueName: \"kubernetes.io/projected/61b89140-235e-4032-ad7f-f18c4a976dcf-kube-api-access-vc98d\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937601 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-system-cni-dir\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937628 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c280d2a-6833-4775-a794-317b88a86303-serviceca\") pod \"node-ca-g2xmq\" (UID: \"5c280d2a-6833-4775-a794-317b88a86303\") " pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:08.937785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937631 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-sysctl-d\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937663 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-system-cni-dir\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937662 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-hostroot\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937661 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-lib-modules\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937696 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-hostroot\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937733 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5bf105fc-1d98-4507-9129-3a5d1027d705-kubelet-config\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937758 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-kubernetes\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937788 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5bf105fc-1d98-4507-9129-3a5d1027d705-kubelet-config\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937785 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f16011-c0a3-42ac-ace9-fa028fc57f76-tmp-dir\") pod \"node-resolver-lpxbp\" (UID: \"62f16011-c0a3-42ac-ace9-fa028fc57f76\") " pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937821 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/04d29c18-d8ce-4440-8438-3e217fb921df-agent-certs\") pod \"konnectivity-agent-vmpdl\" (UID: \"04d29c18-d8ce-4440-8438-3e217fb921df\") " pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937840 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-var-lib-kubelet\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937847 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-kubernetes\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937876 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-var-lib-kubelet\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937882 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-tuned\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937916 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnzzt\" (UniqueName: \"kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt\") pod \"network-check-target-f2dck\" (UID: \"34dba969-9e7c-44f9-838a-1f6c70c66aa7\") " pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937944 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-multus-conf-dir\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937968 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-sys\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.937995 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-multus-cni-dir\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.938651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938020 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4873f3b-f674-41ca-bc4a-44595135d951-cni-binary-copy\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938068 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-multus-conf-dir\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938088 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f16011-c0a3-42ac-ace9-fa028fc57f76-tmp-dir\") pod \"node-resolver-lpxbp\" (UID: \"62f16011-c0a3-42ac-ace9-fa028fc57f76\") " pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938115 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-sys\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938117 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-modprobe-d\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938023 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-modprobe-d\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938136 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c280d2a-6833-4775-a794-317b88a86303-serviceca\") pod \"node-ca-g2xmq\" (UID: \"5c280d2a-6833-4775-a794-317b88a86303\") " pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938170 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-multus-cni-dir\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938162 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-host\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938215 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4873f3b-f674-41ca-bc4a-44595135d951-multus-daemon-config\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938236 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-host\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938242 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-run-netns\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938282 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-run-netns\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938282 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-var-lib-kubelet\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938317 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-var-lib-kubelet\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938330 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdpjr\" (UniqueName: \"kubernetes.io/projected/f4873f3b-f674-41ca-bc4a-44595135d951-kube-api-access-vdpjr\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938357 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-sysctl-conf\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938406 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk42j\" (UniqueName: \"kubernetes.io/projected/5c280d2a-6833-4775-a794-317b88a86303-kube-api-access-jk42j\") pod \"node-ca-g2xmq\" (UID: \"5c280d2a-6833-4775-a794-317b88a86303\") " pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:08.939723 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938430 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-sysconfig\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938452 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-run\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938473 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61b89140-235e-4032-ad7f-f18c4a976dcf-tmp\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938496 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-cnibin\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938521 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-run-k8s-cni-cncf-io\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938557 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-sysctl-conf\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938586 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/04d29c18-d8ce-4440-8438-3e217fb921df-konnectivity-ca\") pod \"konnectivity-agent-vmpdl\" (UID: \"04d29c18-d8ce-4440-8438-3e217fb921df\") " pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938636 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-run\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938691 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-sysconfig\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938731 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-cnibin\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938740 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f4873f3b-f674-41ca-bc4a-44595135d951-multus-daemon-config\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.938788 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f4873f3b-f674-41ca-bc4a-44595135d951-host-run-k8s-cni-cncf-io\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.939320 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/04d29c18-d8ce-4440-8438-3e217fb921df-konnectivity-ca\") pod \"konnectivity-agent-vmpdl\" (UID: \"04d29c18-d8ce-4440-8438-3e217fb921df\") " pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:08.940725 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.940700 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61b89140-235e-4032-ad7f-f18c4a976dcf-etc-tuned\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.941322 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.940775 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/04d29c18-d8ce-4440-8438-3e217fb921df-agent-certs\") pod \"konnectivity-agent-vmpdl\" (UID: \"04d29c18-d8ce-4440-8438-3e217fb921df\") " pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:08.941357 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.941327 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61b89140-235e-4032-ad7f-f18c4a976dcf-tmp\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.945341 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.945322 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:51:08.945432 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.945343 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:51:08.945432 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.945353 2584 projected.go:196] Error preparing data for projected volume kube-api-access-rnzzt for pod openshift-network-diagnostics/network-check-target-f2dck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:08.945432 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:08.945401 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt podName:34dba969-9e7c-44f9-838a-1f6c70c66aa7 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:09.445390264 +0000 UTC m=+3.107964607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rnzzt" (UniqueName: "kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt") pod "network-check-target-f2dck" (UID: "34dba969-9e7c-44f9-838a-1f6c70c66aa7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:08.947500 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.947463 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc98d\" (UniqueName: \"kubernetes.io/projected/61b89140-235e-4032-ad7f-f18c4a976dcf-kube-api-access-vc98d\") pod \"tuned-gb4gp\" (UID: \"61b89140-235e-4032-ad7f-f18c4a976dcf\") " pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:08.947580 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.947480 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q72zj\" (UniqueName: \"kubernetes.io/projected/62f16011-c0a3-42ac-ace9-fa028fc57f76-kube-api-access-q72zj\") pod \"node-resolver-lpxbp\" (UID: \"62f16011-c0a3-42ac-ace9-fa028fc57f76\") " pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:08.949985 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.949968 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdpjr\" (UniqueName: \"kubernetes.io/projected/f4873f3b-f674-41ca-bc4a-44595135d951-kube-api-access-vdpjr\") pod \"multus-867pp\" (UID: \"f4873f3b-f674-41ca-bc4a-44595135d951\") " pod="openshift-multus/multus-867pp" Jul 22 18:51:08.950124 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:08.950105 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk42j\" (UniqueName: \"kubernetes.io/projected/5c280d2a-6833-4775-a794-317b88a86303-kube-api-access-jk42j\") pod \"node-ca-g2xmq\" (UID: \"5c280d2a-6833-4775-a794-317b88a86303\") " pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:09.043571 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.043519 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" Jul 22 18:51:09.059765 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.059736 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hzw9m" Jul 22 18:51:09.066953 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.066924 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" Jul 22 18:51:09.080762 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.080738 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:09.098753 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.098729 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lpxbp" Jul 22 18:51:09.110563 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.110537 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:09.131739 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.131718 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g2xmq" Jul 22 18:51:09.149501 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:09.149457 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2710698bde29e793f9633bc0d191b7e7.slice/crio-9f4e99d897d49447b550c9a0a5e4f3b722b140910208f5e8c0fa441a11d6ea80 WatchSource:0}: Error finding container 9f4e99d897d49447b550c9a0a5e4f3b722b140910208f5e8c0fa441a11d6ea80: Status 404 returned error can't find the container with id 9f4e99d897d49447b550c9a0a5e4f3b722b140910208f5e8c0fa441a11d6ea80 Jul 22 18:51:09.149828 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:09.149792 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6074a94eb729fb72ee8c105664c9c17b.slice/crio-5440f179857265cc9c789a293932791e1beee0e3cc77ab46048cc04aa51e8a3d WatchSource:0}: Error finding container 5440f179857265cc9c789a293932791e1beee0e3cc77ab46048cc04aa51e8a3d: Status 404 returned error can't find the container with id 5440f179857265cc9c789a293932791e1beee0e3cc77ab46048cc04aa51e8a3d Jul 22 18:51:09.150483 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.150469 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" Jul 22 18:51:09.150617 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:09.150593 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c280d2a_6833_4775_a794_317b88a86303.slice/crio-45f113fa4758ce42e65ec2ee355f07562300778b2fbaa8e33fa8f59f58ef2100 WatchSource:0}: Error finding container 45f113fa4758ce42e65ec2ee355f07562300778b2fbaa8e33fa8f59f58ef2100: Status 404 returned error can't find the container with id 45f113fa4758ce42e65ec2ee355f07562300778b2fbaa8e33fa8f59f58ef2100 Jul 22 18:51:09.154292 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.154270 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 18:51:09.158412 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:09.158390 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61b89140_235e_4032_ad7f_f18c4a976dcf.slice/crio-647ca05aca164c69e422c0ccfb4d1bea7a61b30d5254bdcc6f7373ecf5918732 WatchSource:0}: Error finding container 647ca05aca164c69e422c0ccfb4d1bea7a61b30d5254bdcc6f7373ecf5918732: Status 404 returned error can't find the container with id 647ca05aca164c69e422c0ccfb4d1bea7a61b30d5254bdcc6f7373ecf5918732 Jul 22 18:51:09.160766 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.160689 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-867pp" Jul 22 18:51:09.167510 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:09.167490 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4873f3b_f674_41ca_bc4a_44595135d951.slice/crio-80938e4abd1a918564848c24e661ebfc870b44b8be60fb294f310a11918bd704 WatchSource:0}: Error finding container 80938e4abd1a918564848c24e661ebfc870b44b8be60fb294f310a11918bd704: Status 404 returned error can't find the container with id 80938e4abd1a918564848c24e661ebfc870b44b8be60fb294f310a11918bd704 Jul 22 18:51:09.342722 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.342613 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:09.342872 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:09.342791 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:09.342872 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:09.342865 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs podName:d6610a82-7a15-4b2f-9d80-8d2eae96e69e nodeName:}" failed. No retries permitted until 2026-07-22 18:51:10.342846462 +0000 UTC m=+4.005420780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs") pod "network-metrics-daemon-l98h2" (UID: "d6610a82-7a15-4b2f-9d80-8d2eae96e69e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:09.442860 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:09.442829 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d29c18_d8ce_4440_8438_3e217fb921df.slice/crio-0878f36163094b2f1aded19e0871e324abb525ef0a4c453cd54bb7625ac90a96 WatchSource:0}: Error finding container 0878f36163094b2f1aded19e0871e324abb525ef0a4c453cd54bb7625ac90a96: Status 404 returned error can't find the container with id 0878f36163094b2f1aded19e0871e324abb525ef0a4c453cd54bb7625ac90a96 Jul 22 18:51:09.443059 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.443036 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:09.443163 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:09.443148 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:09.443218 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:09.443207 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret podName:5bf105fc-1d98-4507-9129-3a5d1027d705 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:10.44319317 +0000 UTC m=+4.105767492 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret") pod "global-pull-secret-syncer-247b8" (UID: "5bf105fc-1d98-4507-9129-3a5d1027d705") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:09.530251 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:09.530209 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4575c424_46ed_4880_8fec_5198c2a66f3b.slice/crio-55e6000332cabaf5a77c03b2a82863519ef9daf8e07ad164fbdef935ef2b9948 WatchSource:0}: Error finding container 55e6000332cabaf5a77c03b2a82863519ef9daf8e07ad164fbdef935ef2b9948: Status 404 returned error can't find the container with id 55e6000332cabaf5a77c03b2a82863519ef9daf8e07ad164fbdef935ef2b9948 Jul 22 18:51:09.543953 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.543694 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnzzt\" (UniqueName: \"kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt\") pod \"network-check-target-f2dck\" (UID: \"34dba969-9e7c-44f9-838a-1f6c70c66aa7\") " pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:09.543953 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:09.543834 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:51:09.543953 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:09.543860 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:51:09.543953 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:09.543871 2584 projected.go:196] Error preparing data for projected volume kube-api-access-rnzzt for pod openshift-network-diagnostics/network-check-target-f2dck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:09.543953 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:09.543931 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt podName:34dba969-9e7c-44f9-838a-1f6c70c66aa7 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:10.543910218 +0000 UTC m=+4.206484546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnzzt" (UniqueName: "kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt") pod "network-check-target-f2dck" (UID: "34dba969-9e7c-44f9-838a-1f6c70c66aa7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:09.553971 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:09.553934 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad39009_35e7_48ce_bb70_a0b607148a0d.slice/crio-a95d7f2910ade08918afccde07468192dc72982b783159f562432347e4089e35 WatchSource:0}: Error finding container a95d7f2910ade08918afccde07468192dc72982b783159f562432347e4089e35: Status 404 returned error can't find the container with id a95d7f2910ade08918afccde07468192dc72982b783159f562432347e4089e35 Jul 22 18:51:09.651045 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:09.651010 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f16011_c0a3_42ac_ace9_fa028fc57f76.slice/crio-6cc8ccde0a718b007a2eee853f6d10ae0563d6d53e4224fc1cb09abde7cc4c9e WatchSource:0}: Error finding container 6cc8ccde0a718b007a2eee853f6d10ae0563d6d53e4224fc1cb09abde7cc4c9e: Status 404 returned error can't find the container with id 6cc8ccde0a718b007a2eee853f6d10ae0563d6d53e4224fc1cb09abde7cc4c9e Jul 22 18:51:09.769497 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.769426 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 18:46:08 +0000 UTC" deadline="2028-01-25 04:53:27.273958308 +0000 UTC" Jul 22 18:51:09.769497 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.769493 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13234h2m17.504470384s" Jul 22 18:51:09.911459 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.911369 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hzw9m" event={"ID":"0ad39009-35e7-48ce-bb70-a0b607148a0d","Type":"ContainerStarted","Data":"a95d7f2910ade08918afccde07468192dc72982b783159f562432347e4089e35"} Jul 22 18:51:09.915198 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.915165 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" event={"ID":"4575c424-46ed-4880-8fec-5198c2a66f3b","Type":"ContainerStarted","Data":"55e6000332cabaf5a77c03b2a82863519ef9daf8e07ad164fbdef935ef2b9948"} Jul 22 18:51:09.917454 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.917417 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vmpdl" event={"ID":"04d29c18-d8ce-4440-8438-3e217fb921df","Type":"ContainerStarted","Data":"0878f36163094b2f1aded19e0871e324abb525ef0a4c453cd54bb7625ac90a96"} Jul 22 18:51:09.921200 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.921172 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-867pp" event={"ID":"f4873f3b-f674-41ca-bc4a-44595135d951","Type":"ContainerStarted","Data":"80938e4abd1a918564848c24e661ebfc870b44b8be60fb294f310a11918bd704"} Jul 22 18:51:09.923207 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.923170 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" event={"ID":"61b89140-235e-4032-ad7f-f18c4a976dcf","Type":"ContainerStarted","Data":"647ca05aca164c69e422c0ccfb4d1bea7a61b30d5254bdcc6f7373ecf5918732"} Jul 22 18:51:09.925996 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.925970 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g2xmq" event={"ID":"5c280d2a-6833-4775-a794-317b88a86303","Type":"ContainerStarted","Data":"45f113fa4758ce42e65ec2ee355f07562300778b2fbaa8e33fa8f59f58ef2100"} Jul 22 18:51:09.928610 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.928564 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" event={"ID":"6074a94eb729fb72ee8c105664c9c17b","Type":"ContainerStarted","Data":"5440f179857265cc9c789a293932791e1beee0e3cc77ab46048cc04aa51e8a3d"} Jul 22 18:51:09.930766 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.930738 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lpxbp" event={"ID":"62f16011-c0a3-42ac-ace9-fa028fc57f76","Type":"ContainerStarted","Data":"6cc8ccde0a718b007a2eee853f6d10ae0563d6d53e4224fc1cb09abde7cc4c9e"} Jul 22 18:51:09.933948 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.933920 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal" event={"ID":"2710698bde29e793f9633bc0d191b7e7","Type":"ContainerStarted","Data":"9f4e99d897d49447b550c9a0a5e4f3b722b140910208f5e8c0fa441a11d6ea80"} Jul 22 18:51:09.936280 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:09.936253 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" event={"ID":"30190c62-e518-4aad-9d3b-2d75330a372b","Type":"ContainerStarted","Data":"6e313d2b2acbe2a66fa0f039909eb049416163002d1f1e8242284a8a018dbf29"} Jul 22 18:51:10.350933 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:10.350832 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:10.351093 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.351075 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:10.351165 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.351152 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs podName:d6610a82-7a15-4b2f-9d80-8d2eae96e69e nodeName:}" failed. No retries permitted until 2026-07-22 18:51:12.35113099 +0000 UTC m=+6.013705316 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs") pod "network-metrics-daemon-l98h2" (UID: "d6610a82-7a15-4b2f-9d80-8d2eae96e69e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:10.452174 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:10.452135 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:10.452367 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.452320 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:10.452438 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.452387 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret podName:5bf105fc-1d98-4507-9129-3a5d1027d705 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:12.452367133 +0000 UTC m=+6.114941458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret") pod "global-pull-secret-syncer-247b8" (UID: "5bf105fc-1d98-4507-9129-3a5d1027d705") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:10.553257 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:10.553216 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnzzt\" (UniqueName: \"kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt\") pod \"network-check-target-f2dck\" (UID: \"34dba969-9e7c-44f9-838a-1f6c70c66aa7\") " pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:10.553440 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.553402 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:51:10.553440 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.553423 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:51:10.553440 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.553437 2584 projected.go:196] Error preparing data for projected volume kube-api-access-rnzzt for pod openshift-network-diagnostics/network-check-target-f2dck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:10.553637 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.553496 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt podName:34dba969-9e7c-44f9-838a-1f6c70c66aa7 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:12.553477274 +0000 UTC m=+6.216051615 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnzzt" (UniqueName: "kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt") pod "network-check-target-f2dck" (UID: "34dba969-9e7c-44f9-838a-1f6c70c66aa7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:10.770191 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:10.770039 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 18:46:08 +0000 UTC" deadline="2028-03-09 23:27:07.817611203 +0000 UTC" Jul 22 18:51:10.770191 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:10.770145 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14308h35m57.04747175s" Jul 22 18:51:10.903533 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:10.901267 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:10.903533 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.901410 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:10.903533 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:10.901966 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:10.903533 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.902071 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:10.903533 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:10.902184 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:10.903533 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:10.902282 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:10.949567 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:10.949507 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" event={"ID":"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac","Type":"ContainerStarted","Data":"fe37098009643073eebd319b9f185a376247cd3e8ec8d4e98bb2cce704cc0320"} Jul 22 18:51:12.369467 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:12.369433 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:12.370058 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.369578 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:12.370058 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.369647 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs podName:d6610a82-7a15-4b2f-9d80-8d2eae96e69e nodeName:}" failed. No retries permitted until 2026-07-22 18:51:16.369620806 +0000 UTC m=+10.032195140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs") pod "network-metrics-daemon-l98h2" (UID: "d6610a82-7a15-4b2f-9d80-8d2eae96e69e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:12.469946 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:12.469906 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:12.470105 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.470074 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:12.470173 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.470137 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret podName:5bf105fc-1d98-4507-9129-3a5d1027d705 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:16.470122189 +0000 UTC m=+10.132696506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret") pod "global-pull-secret-syncer-247b8" (UID: "5bf105fc-1d98-4507-9129-3a5d1027d705") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:12.570970 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:12.570934 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnzzt\" (UniqueName: \"kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt\") pod \"network-check-target-f2dck\" (UID: \"34dba969-9e7c-44f9-838a-1f6c70c66aa7\") " pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:12.571161 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.571128 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:51:12.571161 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.571155 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:51:12.571293 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.571168 2584 projected.go:196] Error preparing data for projected volume kube-api-access-rnzzt for pod openshift-network-diagnostics/network-check-target-f2dck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:12.571293 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.571229 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt podName:34dba969-9e7c-44f9-838a-1f6c70c66aa7 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:16.571209532 +0000 UTC m=+10.233783851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnzzt" (UniqueName: "kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt") pod "network-check-target-f2dck" (UID: "34dba969-9e7c-44f9-838a-1f6c70c66aa7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:12.901754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:12.900962 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:12.901754 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.901094 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:12.901754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:12.901451 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:12.901754 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.901571 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:12.901754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:12.901640 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:12.901754 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:12.901721 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:14.901033 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:14.900953 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:14.901428 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:14.901108 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:14.901581 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:14.901565 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:14.901725 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:14.901700 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:14.901802 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:14.901771 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:14.901851 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:14.901838 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:16.405201 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:16.405168 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:16.405639 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.405336 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:16.405639 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.405385 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs podName:d6610a82-7a15-4b2f-9d80-8d2eae96e69e nodeName:}" failed. No retries permitted until 2026-07-22 18:51:24.405370609 +0000 UTC m=+18.067944928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs") pod "network-metrics-daemon-l98h2" (UID: "d6610a82-7a15-4b2f-9d80-8d2eae96e69e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:16.505782 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:16.505745 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:16.505942 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.505894 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:16.506016 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.505953 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret podName:5bf105fc-1d98-4507-9129-3a5d1027d705 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:24.505935892 +0000 UTC m=+18.168510224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret") pod "global-pull-secret-syncer-247b8" (UID: "5bf105fc-1d98-4507-9129-3a5d1027d705") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:16.606811 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:16.606764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnzzt\" (UniqueName: \"kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt\") pod \"network-check-target-f2dck\" (UID: \"34dba969-9e7c-44f9-838a-1f6c70c66aa7\") " pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:16.606959 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.606940 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:51:16.607031 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.606959 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:51:16.607031 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.606990 2584 projected.go:196] Error preparing data for projected volume kube-api-access-rnzzt for pod openshift-network-diagnostics/network-check-target-f2dck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:16.607115 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.607053 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt podName:34dba969-9e7c-44f9-838a-1f6c70c66aa7 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:24.607034661 +0000 UTC m=+18.269608990 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnzzt" (UniqueName: "kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt") pod "network-check-target-f2dck" (UID: "34dba969-9e7c-44f9-838a-1f6c70c66aa7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:16.902179 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:16.900684 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:16.902179 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.900809 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:16.902179 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:16.901192 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:16.902179 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.901293 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:16.902487 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:16.902223 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:16.902487 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:16.902307 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:18.901540 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:18.900904 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:18.901540 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:18.900904 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:18.901540 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:18.901020 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:18.901540 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:18.901086 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:18.901540 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:18.900919 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:18.901540 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:18.901178 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:20.904433 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:20.904035 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:20.904433 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:20.904070 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:20.904433 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:20.904136 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:20.904433 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:20.904197 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:20.904433 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:20.904302 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:20.904433 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:20.904403 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:22.901306 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:22.901273 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:22.901755 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:22.901272 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:22.901755 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:22.901409 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:22.901755 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:22.901407 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:22.901755 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:22.901531 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:22.901755 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:22.901620 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:24.470153 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:24.470110 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:24.470591 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.470266 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:24.470591 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.470335 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs podName:d6610a82-7a15-4b2f-9d80-8d2eae96e69e nodeName:}" failed. No retries permitted until 2026-07-22 18:51:40.470318019 +0000 UTC m=+34.132892339 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs") pod "network-metrics-daemon-l98h2" (UID: "d6610a82-7a15-4b2f-9d80-8d2eae96e69e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:24.570648 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:24.570608 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:24.570813 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.570776 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:24.570878 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.570853 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret podName:5bf105fc-1d98-4507-9129-3a5d1027d705 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:40.570832874 +0000 UTC m=+34.233407195 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret") pod "global-pull-secret-syncer-247b8" (UID: "5bf105fc-1d98-4507-9129-3a5d1027d705") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:24.671060 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:24.670964 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnzzt\" (UniqueName: \"kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt\") pod \"network-check-target-f2dck\" (UID: \"34dba969-9e7c-44f9-838a-1f6c70c66aa7\") " pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:24.671205 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.671128 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:51:24.671205 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.671150 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:51:24.671205 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.671162 2584 projected.go:196] Error preparing data for projected volume kube-api-access-rnzzt for pod openshift-network-diagnostics/network-check-target-f2dck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:24.671304 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.671214 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt podName:34dba969-9e7c-44f9-838a-1f6c70c66aa7 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:40.671199925 +0000 UTC m=+34.333774242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnzzt" (UniqueName: "kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt") pod "network-check-target-f2dck" (UID: "34dba969-9e7c-44f9-838a-1f6c70c66aa7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:24.901434 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:24.900974 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:24.901434 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:24.900996 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:24.901434 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:24.900984 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:24.901434 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.901094 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:24.901434 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.901192 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:24.901434 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:24.901286 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:26.901041 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:26.901005 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:26.901549 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:26.901114 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:26.901549 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:26.901146 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:26.901549 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:26.901116 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:26.901549 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:26.901243 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:26.901549 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:26.901343 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:28.900534 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:28.900490 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:28.900534 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:28.900534 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:28.901076 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:28.900590 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:28.901076 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:28.900697 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:28.901076 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:28.900797 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:28.901076 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:28.900870 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:29.987139 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:29.986882 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" event={"ID":"61b89140-235e-4032-ad7f-f18c4a976dcf","Type":"ContainerStarted","Data":"6fcfa08e2007bf8bda9adc7da0d2712328e9ca705824b6ee5c02e1646e4f3c02"} Jul 22 18:51:29.988406 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:29.988383 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g2xmq" event={"ID":"5c280d2a-6833-4775-a794-317b88a86303","Type":"ContainerStarted","Data":"7451f25a27f4deb34abbdbc7c4321bdacaf9bec6d1d67c1846dde548a77097bd"} Jul 22 18:51:29.989842 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:29.989814 2584 generic.go:356] "Generic (PLEG): container finished" podID="6074a94eb729fb72ee8c105664c9c17b" containerID="dea9884d19845a4617e30d93a69a140f11235dd6407755832aab16893cdb2c1f" exitCode=0 Jul 22 18:51:29.990012 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:29.989891 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" event={"ID":"6074a94eb729fb72ee8c105664c9c17b","Type":"ContainerDied","Data":"dea9884d19845a4617e30d93a69a140f11235dd6407755832aab16893cdb2c1f"} Jul 22 18:51:29.991389 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:29.991362 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lpxbp" event={"ID":"62f16011-c0a3-42ac-ace9-fa028fc57f76","Type":"ContainerStarted","Data":"903ab97a333aa0011a54007cdf09af348aa51945402087828c782cf4a7df4d8a"} Jul 22 18:51:29.992839 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:29.992816 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal" event={"ID":"2710698bde29e793f9633bc0d191b7e7","Type":"ContainerStarted","Data":"7f82a17e76a9e7e1ce967473f7c50e33449a77409dc0865086c2c78686a020db"} Jul 22 18:51:29.994120 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:29.994099 2584 generic.go:356] "Generic (PLEG): container finished" podID="124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac" containerID="e0de85a458feedd79d128c194c6ab28315e66d58abe6727787b9e7380d313f27" exitCode=0 Jul 22 18:51:29.994221 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:29.994127 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" event={"ID":"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac","Type":"ContainerDied","Data":"e0de85a458feedd79d128c194c6ab28315e66d58abe6727787b9e7380d313f27"} Jul 22 18:51:29.998700 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:29.998669 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" event={"ID":"30190c62-e518-4aad-9d3b-2d75330a372b","Type":"ContainerStarted","Data":"545e7264d67a5d7f8cb99496987b4a9c42397e71e0d9f5826f8a1851fb80ff03"} Jul 22 18:51:30.001468 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.001448 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" event={"ID":"4575c424-46ed-4880-8fec-5198c2a66f3b","Type":"ContainerStarted","Data":"fe0460928f4750371a4c564050427c0559f74de02df07cd7265097f01277787a"} Jul 22 18:51:30.001582 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.001511 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" event={"ID":"4575c424-46ed-4880-8fec-5198c2a66f3b","Type":"ContainerStarted","Data":"3f44694b77dfdb44c1f0683df10c7d524650d179d5ab7f3b7e6600b5e8cd24d4"} Jul 22 18:51:30.001582 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.001563 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" event={"ID":"4575c424-46ed-4880-8fec-5198c2a66f3b","Type":"ContainerStarted","Data":"e5e74bf075ae76f8166489e0beec6e31b5dbe7a3d89a13ce928b536328d3642c"} Jul 22 18:51:30.001672 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.001608 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" event={"ID":"4575c424-46ed-4880-8fec-5198c2a66f3b","Type":"ContainerStarted","Data":"5e3a1af747286c8d19f7291c97b44a23f5072024123b481c56e59bd210bea71b"} Jul 22 18:51:30.001672 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.001649 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" event={"ID":"4575c424-46ed-4880-8fec-5198c2a66f3b","Type":"ContainerStarted","Data":"1b47cebd71e0671fad68beaedcedb21436338c390130af89ff10c0f2f56d7ed4"} Jul 22 18:51:30.001743 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.001688 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" event={"ID":"4575c424-46ed-4880-8fec-5198c2a66f3b","Type":"ContainerStarted","Data":"71ebafd84681325f0c35d58f858049d0fe21e7d96f8ce6208b95e2b1a2a25daa"} Jul 22 18:51:30.002776 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.002760 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vmpdl" event={"ID":"04d29c18-d8ce-4440-8438-3e217fb921df","Type":"ContainerStarted","Data":"e1b2fdb08f2dd9725052defc321b3bf6f9ef3d39f49ce7fea5981033516b27c6"} Jul 22 18:51:30.004111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.004090 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-867pp" event={"ID":"f4873f3b-f674-41ca-bc4a-44595135d951","Type":"ContainerStarted","Data":"69482eac010c24fe651f5f18f0d43cc57765dfdc8d54904d05813e8b6eca4bfc"} Jul 22 18:51:30.004382 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.004263 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gb4gp" podStartSLOduration=3.131149518 podStartE2EDuration="23.004249958s" totalImagesPullingTime="19.87310044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:30.00410349 +0000 UTC m=+23.666677829" watchObservedRunningTime="2026-07-22 18:51:30.004249958 +0000 UTC m=+23.666824289" Jul 22 18:51:30.019565 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.019499 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vmpdl" podStartSLOduration=3.431196389 podStartE2EDuration="23.019479229s" totalImagesPullingTime="19.58828284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:30.01930672 +0000 UTC m=+23.681881061" watchObservedRunningTime="2026-07-22 18:51:30.019479229 +0000 UTC m=+23.682053571" Jul 22 18:51:30.070494 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.070453 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lpxbp" podStartSLOduration=3.695558585 podStartE2EDuration="23.070438027s" totalImagesPullingTime="19.374879442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:30.070100013 +0000 UTC m=+23.732674351" watchObservedRunningTime="2026-07-22 18:51:30.070438027 +0000 UTC m=+23.733012366" Jul 22 18:51:30.090220 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.090178 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g2xmq" podStartSLOduration=3.211486541 podStartE2EDuration="23.090160123s" totalImagesPullingTime="19.878673582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:30.089761278 +0000 UTC m=+23.752335616" watchObservedRunningTime="2026-07-22 18:51:30.090160123 +0000 UTC m=+23.752734462" Jul 22 18:51:30.126036 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.125987 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-177.ec2.internal" podStartSLOduration=22.125974501 podStartE2EDuration="22.125974501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:30.103334415 +0000 UTC m=+23.765908753" watchObservedRunningTime="2026-07-22 18:51:30.125974501 +0000 UTC m=+23.788548839" Jul 22 18:51:30.126235 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.126212 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-867pp" podStartSLOduration=3.217833309 podStartE2EDuration="23.126206196s" totalImagesPullingTime="19.908372887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:30.125673324 +0000 UTC m=+23.788247683" watchObservedRunningTime="2026-07-22 18:51:30.126206196 +0000 UTC m=+23.788780536" Jul 22 18:51:30.902559 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.902467 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:30.902807 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:30.902613 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:30.902807 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.902689 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:30.902807 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:30.902735 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:30.902807 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:30.902777 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:30.903052 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:30.902823 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:31.008013 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:31.007974 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hzw9m" event={"ID":"0ad39009-35e7-48ce-bb70-a0b607148a0d","Type":"ContainerStarted","Data":"461b93790607dba33ae63713cdb951ad5fa6445f84dbc7e5efb3a3a80e279eb0"} Jul 22 18:51:31.012070 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:31.012043 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" event={"ID":"6074a94eb729fb72ee8c105664c9c17b","Type":"ContainerStarted","Data":"0d41092e301c63007e4884b03cce98b09f2eef4f1b9b884a3b90635ca8eb384b"} Jul 22 18:51:31.027790 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:31.027723 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hzw9m" podStartSLOduration=9.221077235 podStartE2EDuration="24.027704932s" totalImagesPullingTime="14.806627697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:31.027319284 +0000 UTC m=+24.689893623" watchObservedRunningTime="2026-07-22 18:51:31.027704932 +0000 UTC m=+24.690279273" Jul 22 18:51:31.043812 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:31.043757 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-177.ec2.internal" podStartSLOduration=23.043743218 podStartE2EDuration="23.043743218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:31.043385116 +0000 UTC m=+24.705959466" watchObservedRunningTime="2026-07-22 18:51:31.043743218 +0000 UTC m=+24.706317556" Jul 22 18:51:31.664143 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:31.664117 2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 18:51:31.804308 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:31.804167 2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T18:51:31.664139316Z","UUID":"53a0f2c0-2ab2-457b-be34-3de28d0f5e39","Handler":null,"Name":"","Endpoint":""} Jul 22 18:51:31.805826 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:31.805799 2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 18:51:31.805967 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:31.805842 2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 18:51:32.015421 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:32.015389 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" event={"ID":"30190c62-e518-4aad-9d3b-2d75330a372b","Type":"ContainerStarted","Data":"54cf1a759932d6e09b3921c4897585577b6f4c87a3eaac49ddeb17f53b850583"} Jul 22 18:51:32.900755 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:32.900719 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:32.901015 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:32.900850 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:32.901015 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:32.900734 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:32.901015 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:32.900940 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:32.901015 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:32.900725 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:32.901257 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:32.901036 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:33.021496 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:33.021459 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" event={"ID":"4575c424-46ed-4880-8fec-5198c2a66f3b","Type":"ContainerStarted","Data":"c1f44d4967688e5d62fe11c2959c9730f812cc8f0d423eef5f91ab5248724c54"} Jul 22 18:51:34.876450 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:34.876253 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:34.877156 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:34.876921 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:34.900178 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:34.900157 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:34.900178 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:34.900172 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:34.900294 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:34.900241 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:34.900340 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:34.900290 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:34.900374 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:34.900341 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:34.900421 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:34.900390 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:35.027255 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.027231 2584 generic.go:356] "Generic (PLEG): container finished" podID="124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac" containerID="798cbcadba0025c7affc18e31d65d2356885edab245ba8f025451b2fbd0a8e53" exitCode=0 Jul 22 18:51:35.027412 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.027294 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" event={"ID":"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac","Type":"ContainerDied","Data":"798cbcadba0025c7affc18e31d65d2356885edab245ba8f025451b2fbd0a8e53"} Jul 22 18:51:35.029569 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.029549 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" event={"ID":"30190c62-e518-4aad-9d3b-2d75330a372b","Type":"ContainerStarted","Data":"e0bb0280df6ec19bab61f71c018714b454e43041e4ee33d1982f29a07c0e58cb"} Jul 22 18:51:35.033367 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.033336 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" event={"ID":"4575c424-46ed-4880-8fec-5198c2a66f3b","Type":"ContainerStarted","Data":"d9f24812d4ac8d86ae315211aae88240d12020ab84008d81f3ba3cee7c5b67b8"} Jul 22 18:51:35.033588 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.033562 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:35.033690 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.033646 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:35.033690 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.033687 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:35.049416 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.049393 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:35.049594 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.049570 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:51:35.073313 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.073278 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" podStartSLOduration=8.44258477 podStartE2EDuration="28.07326653s" totalImagesPullingTime="19.63068176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:35.07297795 +0000 UTC m=+28.735552300" watchObservedRunningTime="2026-07-22 18:51:35.07326653 +0000 UTC m=+28.735840869" Jul 22 18:51:35.092606 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:35.092570 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tbmfs" podStartSLOduration=8.671197424 podStartE2EDuration="28.092559202s" totalImagesPullingTime="19.421361778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:35.092422001 +0000 UTC m=+28.754996340" watchObservedRunningTime="2026-07-22 18:51:35.092559202 +0000 UTC m=+28.755133537" Jul 22 18:51:36.348602 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:36.348323 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l98h2"] Jul 22 18:51:36.349037 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:36.348703 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:36.349037 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:36.348832 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:36.351200 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:36.351175 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f2dck"] Jul 22 18:51:36.351330 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:36.351262 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:36.351389 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:36.351338 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:36.351804 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:36.351784 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-247b8"] Jul 22 18:51:36.351908 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:36.351877 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:36.351986 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:36.351967 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:37.040648 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:37.040611 2584 generic.go:356] "Generic (PLEG): container finished" podID="124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac" containerID="0cad8c83615781d8bfdf974d63e8b8600a6af1690cac6c40bbcaa76a70ac6923" exitCode=0 Jul 22 18:51:37.040830 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:37.040688 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" event={"ID":"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac","Type":"ContainerDied","Data":"0cad8c83615781d8bfdf974d63e8b8600a6af1690cac6c40bbcaa76a70ac6923"} Jul 22 18:51:37.425860 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:37.425788 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:37.426236 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:37.425887 2584 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 22 18:51:37.426271 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:37.426256 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vmpdl" Jul 22 18:51:37.900224 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:37.900149 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:37.900387 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:37.900247 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:37.900387 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:37.900252 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:37.900387 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:37.900266 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:37.900387 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:37.900359 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:37.900591 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:37.900443 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:39.901207 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:39.901034 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:39.901688 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:39.901036 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:39.901688 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:39.901276 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:39.901688 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:39.901086 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:39.901688 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:39.901373 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:39.901688 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:39.901436 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:40.051298 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:40.051268 2584 generic.go:356] "Generic (PLEG): container finished" podID="124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac" containerID="ac7650b57a88f034968da89befa871cdbc108c5b97c7e4e2ed5fb1a851dc6b1b" exitCode=0 Jul 22 18:51:40.051441 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:40.051305 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" event={"ID":"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac","Type":"ContainerDied","Data":"ac7650b57a88f034968da89befa871cdbc108c5b97c7e4e2ed5fb1a851dc6b1b"} Jul 22 18:51:40.489792 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:40.489761 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:40.489951 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:40.489877 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:40.489951 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:40.489928 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs podName:d6610a82-7a15-4b2f-9d80-8d2eae96e69e nodeName:}" failed. No retries permitted until 2026-07-22 18:52:12.489914188 +0000 UTC m=+66.152488508 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs") pod "network-metrics-daemon-l98h2" (UID: "d6610a82-7a15-4b2f-9d80-8d2eae96e69e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:51:40.590351 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:40.590307 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:40.590507 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:40.590421 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:40.590507 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:40.590472 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret podName:5bf105fc-1d98-4507-9129-3a5d1027d705 nodeName:}" failed. No retries permitted until 2026-07-22 18:52:12.590458898 +0000 UTC m=+66.253033216 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret") pod "global-pull-secret-syncer-247b8" (UID: "5bf105fc-1d98-4507-9129-3a5d1027d705") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:51:40.691149 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:40.691118 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnzzt\" (UniqueName: \"kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt\") pod \"network-check-target-f2dck\" (UID: \"34dba969-9e7c-44f9-838a-1f6c70c66aa7\") " pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:40.691282 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:40.691229 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:51:40.691282 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:40.691242 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:51:40.691282 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:40.691250 2584 projected.go:196] Error preparing data for projected volume kube-api-access-rnzzt for pod openshift-network-diagnostics/network-check-target-f2dck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:40.691378 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:40.691295 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt podName:34dba969-9e7c-44f9-838a-1f6c70c66aa7 nodeName:}" failed. No retries permitted until 2026-07-22 18:52:12.691282004 +0000 UTC m=+66.353856326 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rnzzt" (UniqueName: "kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt") pod "network-check-target-f2dck" (UID: "34dba969-9e7c-44f9-838a-1f6c70c66aa7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:51:41.900923 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:41.900847 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:41.901315 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:41.900967 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:51:41.901315 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:41.901012 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:41.901315 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:41.901027 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:41.901315 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:41.901066 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2dck" podUID="34dba969-9e7c-44f9-838a-1f6c70c66aa7" Jul 22 18:51:41.901315 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:41.901120 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-247b8" podUID="5bf105fc-1d98-4507-9129-3a5d1027d705" Jul 22 18:51:42.160823 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.160747 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-177.ec2.internal" event="NodeReady" Jul 22 18:51:42.160978 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.160895 2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 18:51:42.206581 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.206549 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7lk8l"] Jul 22 18:51:42.209152 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.209115 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7qjbt"] Jul 22 18:51:42.209283 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.209252 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.210853 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.210828 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7qjbt" Jul 22 18:51:42.211728 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.211707 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-phz5q\"" Jul 22 18:51:42.211928 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.211907 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 18:51:42.212022 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.211957 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 18:51:42.213398 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.213364 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 18:51:42.213507 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.213467 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 18:51:42.213613 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.213503 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 18:51:42.213691 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.213650 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wjx5r\"" Jul 22 18:51:42.221832 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.221810 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7lk8l"] Jul 22 18:51:42.223544 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.223481 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7qjbt"] Jul 22 18:51:42.303915 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.303887 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gd4b\" (UniqueName: \"kubernetes.io/projected/b6144392-e95f-4cf4-973c-f73ce61a0780-kube-api-access-7gd4b\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.304057 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.303953 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6144392-e95f-4cf4-973c-f73ce61a0780-tmp-dir\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.304108 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.304056 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.304108 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.304089 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l27rw\" (UniqueName: \"kubernetes.io/projected/5037332a-7cf0-489d-b14c-ce356a5a3c4d-kube-api-access-l27rw\") pod \"ingress-canary-7qjbt\" (UID: \"5037332a-7cf0-489d-b14c-ce356a5a3c4d\") " pod="openshift-ingress-canary/ingress-canary-7qjbt" Jul 22 18:51:42.304214 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.304110 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5037332a-7cf0-489d-b14c-ce356a5a3c4d-cert\") pod \"ingress-canary-7qjbt\" (UID: \"5037332a-7cf0-489d-b14c-ce356a5a3c4d\") " pod="openshift-ingress-canary/ingress-canary-7qjbt" Jul 22 18:51:42.304214 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.304154 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b6144392-e95f-4cf4-973c-f73ce61a0780-config-volume\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.405289 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.405255 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.405451 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.405313 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l27rw\" (UniqueName: \"kubernetes.io/projected/5037332a-7cf0-489d-b14c-ce356a5a3c4d-kube-api-access-l27rw\") pod \"ingress-canary-7qjbt\" (UID: \"5037332a-7cf0-489d-b14c-ce356a5a3c4d\") " pod="openshift-ingress-canary/ingress-canary-7qjbt" Jul 22 18:51:42.405451 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.405344 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5037332a-7cf0-489d-b14c-ce356a5a3c4d-cert\") pod \"ingress-canary-7qjbt\" (UID: \"5037332a-7cf0-489d-b14c-ce356a5a3c4d\") " pod="openshift-ingress-canary/ingress-canary-7qjbt" Jul 22 18:51:42.405451 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.405368 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b6144392-e95f-4cf4-973c-f73ce61a0780-config-volume\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.405451 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.405419 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gd4b\" (UniqueName: \"kubernetes.io/projected/b6144392-e95f-4cf4-973c-f73ce61a0780-kube-api-access-7gd4b\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.405451 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:42.405435 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:51:42.406011 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.405461 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6144392-e95f-4cf4-973c-f73ce61a0780-tmp-dir\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.406011 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:42.405546 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls podName:b6144392-e95f-4cf4-973c-f73ce61a0780 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:42.905506276 +0000 UTC m=+36.568080613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls") pod "dns-default-7lk8l" (UID: "b6144392-e95f-4cf4-973c-f73ce61a0780") : secret "dns-default-metrics-tls" not found Jul 22 18:51:42.406011 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.405854 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6144392-e95f-4cf4-973c-f73ce61a0780-tmp-dir\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.406182 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.406130 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b6144392-e95f-4cf4-973c-f73ce61a0780-config-volume\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.410978 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.410899 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5037332a-7cf0-489d-b14c-ce356a5a3c4d-cert\") pod \"ingress-canary-7qjbt\" (UID: \"5037332a-7cf0-489d-b14c-ce356a5a3c4d\") " pod="openshift-ingress-canary/ingress-canary-7qjbt" Jul 22 18:51:42.415376 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.415351 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gd4b\" (UniqueName: \"kubernetes.io/projected/b6144392-e95f-4cf4-973c-f73ce61a0780-kube-api-access-7gd4b\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.415769 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.415748 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l27rw\" (UniqueName: \"kubernetes.io/projected/5037332a-7cf0-489d-b14c-ce356a5a3c4d-kube-api-access-l27rw\") pod \"ingress-canary-7qjbt\" (UID: \"5037332a-7cf0-489d-b14c-ce356a5a3c4d\") " pod="openshift-ingress-canary/ingress-canary-7qjbt" Jul 22 18:51:42.529991 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.529953 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7qjbt" Jul 22 18:51:42.687998 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.687919 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7qjbt"] Jul 22 18:51:42.692699 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:51:42.692667 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5037332a_7cf0_489d_b14c_ce356a5a3c4d.slice/crio-7fd10bbe543657bf9f8c5037f4f95a842e2f7a5eded1381115b11e6953a05954 WatchSource:0}: Error finding container 7fd10bbe543657bf9f8c5037f4f95a842e2f7a5eded1381115b11e6953a05954: Status 404 returned error can't find the container with id 7fd10bbe543657bf9f8c5037f4f95a842e2f7a5eded1381115b11e6953a05954 Jul 22 18:51:42.909004 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:42.908968 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:42.909678 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:42.909114 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:51:42.909678 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:42.909181 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls podName:b6144392-e95f-4cf4-973c-f73ce61a0780 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:43.909160005 +0000 UTC m=+37.571734322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls") pod "dns-default-7lk8l" (UID: "b6144392-e95f-4cf4-973c-f73ce61a0780") : secret "dns-default-metrics-tls" not found Jul 22 18:51:43.060335 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.060106 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7qjbt" event={"ID":"5037332a-7cf0-489d-b14c-ce356a5a3c4d","Type":"ContainerStarted","Data":"7fd10bbe543657bf9f8c5037f4f95a842e2f7a5eded1381115b11e6953a05954"} Jul 22 18:51:43.900582 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.900548 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:51:43.900582 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.900586 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:51:43.900809 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.900589 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:51:43.903471 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.903451 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 18:51:43.904433 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.904411 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vbtbq\"" Jul 22 18:51:43.904433 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.904426 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 18:51:43.904625 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.904416 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 18:51:43.904625 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.904482 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pz9g5\"" Jul 22 18:51:43.904625 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.904484 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 18:51:43.916651 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:43.916630 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:43.916998 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:43.916712 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:51:43.916998 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:43.916765 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls podName:b6144392-e95f-4cf4-973c-f73ce61a0780 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:45.916750439 +0000 UTC m=+39.579324756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls") pod "dns-default-7lk8l" (UID: "b6144392-e95f-4cf4-973c-f73ce61a0780") : secret "dns-default-metrics-tls" not found Jul 22 18:51:45.068130 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:45.068097 2584 generic.go:356] "Generic (PLEG): container finished" podID="124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac" containerID="b0c67be97f06050bd5a19cceeaae4285b98fa4da93723f7a113e499733341610" exitCode=0 Jul 22 18:51:45.068554 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:45.068149 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" event={"ID":"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac","Type":"ContainerDied","Data":"b0c67be97f06050bd5a19cceeaae4285b98fa4da93723f7a113e499733341610"} Jul 22 18:51:45.934066 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:45.934030 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:45.934228 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:45.934210 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:51:45.934299 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:45.934283 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls podName:b6144392-e95f-4cf4-973c-f73ce61a0780 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:49.93425982 +0000 UTC m=+43.596834141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls") pod "dns-default-7lk8l" (UID: "b6144392-e95f-4cf4-973c-f73ce61a0780") : secret "dns-default-metrics-tls" not found Jul 22 18:51:46.074722 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:46.074678 2584 generic.go:356] "Generic (PLEG): container finished" podID="124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac" containerID="9fce276eca7f582be83f00a8cc1135afbdf77036eff9cc51526d95f6de048e3d" exitCode=0 Jul 22 18:51:46.075149 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:46.074755 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" event={"ID":"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac","Type":"ContainerDied","Data":"9fce276eca7f582be83f00a8cc1135afbdf77036eff9cc51526d95f6de048e3d"} Jul 22 18:51:47.080647 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:47.080611 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" event={"ID":"124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac","Type":"ContainerStarted","Data":"7795de3d5c2aebf672806fe6aa17be9e076bf09b537fa901863c61c4945d62dc"} Jul 22 18:51:47.081930 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:47.081901 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7qjbt" event={"ID":"5037332a-7cf0-489d-b14c-ce356a5a3c4d","Type":"ContainerStarted","Data":"6c8069e5f7a1b8615fbfaf44c2f0b2c8ba3cb1972e18dc868fb6847b2d0446ff"} Jul 22 18:51:47.104789 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:47.104754 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bh8z5" podStartSLOduration=9.111972498 podStartE2EDuration="40.104740104s" totalImagesPullingTime="30.992767606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:47.103712898 +0000 UTC m=+40.766287229" watchObservedRunningTime="2026-07-22 18:51:47.104740104 +0000 UTC m=+40.767314443" Jul 22 18:51:47.120458 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:47.120412 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7qjbt" podStartSLOduration=1.437856454 podStartE2EDuration="5.120398215s" totalImagesPullingTime="3.682541761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:47.120340892 +0000 UTC m=+40.782915231" watchObservedRunningTime="2026-07-22 18:51:47.120398215 +0000 UTC m=+40.782972553" Jul 22 18:51:49.962701 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:49.962660 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:49.963083 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:49.962777 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:51:49.963083 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:49.962844 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls podName:b6144392-e95f-4cf4-973c-f73ce61a0780 nodeName:}" failed. No retries permitted until 2026-07-22 18:51:57.962827714 +0000 UTC m=+51.625402031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls") pod "dns-default-7lk8l" (UID: "b6144392-e95f-4cf4-973c-f73ce61a0780") : secret "dns-default-metrics-tls" not found Jul 22 18:51:58.014889 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:51:58.014846 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:51:58.015349 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:58.014992 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:51:58.015349 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:51:58.015065 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls podName:b6144392-e95f-4cf4-973c-f73ce61a0780 nodeName:}" failed. No retries permitted until 2026-07-22 18:52:14.0150497 +0000 UTC m=+67.677624019 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls") pod "dns-default-7lk8l" (UID: "b6144392-e95f-4cf4-973c-f73ce61a0780") : secret "dns-default-metrics-tls" not found Jul 22 18:52:07.051130 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:07.051101 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xg4p5" Jul 22 18:52:12.518338 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.518292 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:52:12.521091 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.521072 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 18:52:12.529140 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:52:12.529122 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 18:52:12.529231 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:52:12.529186 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs podName:d6610a82-7a15-4b2f-9d80-8d2eae96e69e nodeName:}" failed. No retries permitted until 2026-07-22 18:53:16.529171509 +0000 UTC m=+130.191745827 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs") pod "network-metrics-daemon-l98h2" (UID: "d6610a82-7a15-4b2f-9d80-8d2eae96e69e") : secret "metrics-daemon-secret" not found Jul 22 18:52:12.618803 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.618776 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:52:12.621486 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.621472 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 18:52:12.633266 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.633243 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5bf105fc-1d98-4507-9129-3a5d1027d705-original-pull-secret\") pod \"global-pull-secret-syncer-247b8\" (UID: \"5bf105fc-1d98-4507-9129-3a5d1027d705\") " pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:52:12.712711 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.712679 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-247b8" Jul 22 18:52:12.719644 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.719624 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnzzt\" (UniqueName: \"kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt\") pod \"network-check-target-f2dck\" (UID: \"34dba969-9e7c-44f9-838a-1f6c70c66aa7\") " pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:52:12.722435 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.722412 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 18:52:12.731835 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.731815 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 18:52:12.743339 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.743317 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnzzt\" (UniqueName: \"kubernetes.io/projected/34dba969-9e7c-44f9-838a-1f6c70c66aa7-kube-api-access-rnzzt\") pod \"network-check-target-f2dck\" (UID: \"34dba969-9e7c-44f9-838a-1f6c70c66aa7\") " pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:52:12.830754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:12.830728 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-247b8"] Jul 22 18:52:12.833867 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:52:12.833839 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf105fc_1d98_4507_9129_3a5d1027d705.slice/crio-a28e848cebfaf0f817a5c721d05566add38b38d63b851d717cddb98771347995 WatchSource:0}: Error finding container a28e848cebfaf0f817a5c721d05566add38b38d63b851d717cddb98771347995: Status 404 returned error can't find the container with id a28e848cebfaf0f817a5c721d05566add38b38d63b851d717cddb98771347995 Jul 22 18:52:13.028510 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:13.028483 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vbtbq\"" Jul 22 18:52:13.036526 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:13.036500 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:52:13.140419 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:13.140389 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-247b8" event={"ID":"5bf105fc-1d98-4507-9129-3a5d1027d705","Type":"ContainerStarted","Data":"a28e848cebfaf0f817a5c721d05566add38b38d63b851d717cddb98771347995"} Jul 22 18:52:13.153711 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:13.153682 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f2dck"] Jul 22 18:52:13.156497 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:52:13.156475 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34dba969_9e7c_44f9_838a_1f6c70c66aa7.slice/crio-39cc10710770da98d8ef30c914aad2ea1acd02d8447b5d8bd33cac26297dba99 WatchSource:0}: Error finding container 39cc10710770da98d8ef30c914aad2ea1acd02d8447b5d8bd33cac26297dba99: Status 404 returned error can't find the container with id 39cc10710770da98d8ef30c914aad2ea1acd02d8447b5d8bd33cac26297dba99 Jul 22 18:52:14.028938 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:14.028890 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:52:14.029350 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:52:14.029014 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:52:14.029350 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:52:14.029065 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls podName:b6144392-e95f-4cf4-973c-f73ce61a0780 nodeName:}" failed. No retries permitted until 2026-07-22 18:52:46.02905 +0000 UTC m=+99.691624317 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls") pod "dns-default-7lk8l" (UID: "b6144392-e95f-4cf4-973c-f73ce61a0780") : secret "dns-default-metrics-tls" not found Jul 22 18:52:14.144813 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:14.144771 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f2dck" event={"ID":"34dba969-9e7c-44f9-838a-1f6c70c66aa7","Type":"ContainerStarted","Data":"39cc10710770da98d8ef30c914aad2ea1acd02d8447b5d8bd33cac26297dba99"} Jul 22 18:52:18.156732 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:18.156694 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-247b8" event={"ID":"5bf105fc-1d98-4507-9129-3a5d1027d705","Type":"ContainerStarted","Data":"7720206a253bfcc8d1fefb319114e0e09444340b741a72a080fe94fb581479bd"} Jul 22 18:52:18.158235 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:18.158198 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f2dck" event={"ID":"34dba969-9e7c-44f9-838a-1f6c70c66aa7","Type":"ContainerStarted","Data":"2ef7b5b7378d3007ab45357a9232ee6c67f45597bda677ef14337c2ec84ee6a4"} Jul 22 18:52:18.158337 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:18.158257 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:52:18.173794 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:18.173749 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-247b8" podStartSLOduration=66.237008883 podStartE2EDuration="1m11.17373607s" totalImagesPullingTime="4.936727187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:18.173304251 +0000 UTC m=+71.835878591" watchObservedRunningTime="2026-07-22 18:52:18.17373607 +0000 UTC m=+71.836310409" Jul 22 18:52:18.188691 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:18.188656 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f2dck" podStartSLOduration=66.578200648 podStartE2EDuration="1m11.188643652s" totalImagesPullingTime="4.610443004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:18.188616376 +0000 UTC m=+71.851190715" watchObservedRunningTime="2026-07-22 18:52:18.188643652 +0000 UTC m=+71.851217991" Jul 22 18:52:46.034779 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:46.034737 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:52:46.035137 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:52:46.034891 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:52:46.035137 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:52:46.034958 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls podName:b6144392-e95f-4cf4-973c-f73ce61a0780 nodeName:}" failed. No retries permitted until 2026-07-22 18:53:50.034941705 +0000 UTC m=+163.697516023 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls") pod "dns-default-7lk8l" (UID: "b6144392-e95f-4cf4-973c-f73ce61a0780") : secret "dns-default-metrics-tls" not found Jul 22 18:52:49.163100 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:52:49.163068 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f2dck" Jul 22 18:53:16.625775 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:53:16.625733 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:53:16.626237 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:53:16.625850 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 18:53:16.626237 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:53:16.625916 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs podName:d6610a82-7a15-4b2f-9d80-8d2eae96e69e nodeName:}" failed. No retries permitted until 2026-07-22 18:55:18.625902548 +0000 UTC m=+252.288476867 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs") pod "network-metrics-daemon-l98h2" (UID: "d6610a82-7a15-4b2f-9d80-8d2eae96e69e") : secret "metrics-daemon-secret" not found Jul 22 18:53:45.222841 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:53:45.222802 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7lk8l" podUID="b6144392-e95f-4cf4-973c-f73ce61a0780" Jul 22 18:53:45.362927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:53:45.362903 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lk8l" Jul 22 18:53:46.919847 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:53:46.919797 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:53:50.037143 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:53:50.037094 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:53:50.037603 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:53:50.037251 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:53:50.037603 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:53:50.037340 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls podName:b6144392-e95f-4cf4-973c-f73ce61a0780 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:52.037324092 +0000 UTC m=+285.699898409 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls") pod "dns-default-7lk8l" (UID: "b6144392-e95f-4cf4-973c-f73ce61a0780") : secret "dns-default-metrics-tls" not found Jul 22 18:54:01.900275 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:54:01.900183 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:55:18.712340 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:18.712302 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:55:18.712927 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:18.712417 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 18:55:18.712927 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:18.712477 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs podName:d6610a82-7a15-4b2f-9d80-8d2eae96e69e nodeName:}" failed. No retries permitted until 2026-07-22 18:57:20.712463226 +0000 UTC m=+374.375037543 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs") pod "network-metrics-daemon-l98h2" (UID: "d6610a82-7a15-4b2f-9d80-8d2eae96e69e") : secret "metrics-daemon-secret" not found Jul 22 18:55:20.993600 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.993560 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh"] Jul 22 18:55:20.995096 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.995081 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm"] Jul 22 18:55:20.995220 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.995200 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:20.996907 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.996888 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-lqrlz"] Jul 22 18:55:20.997022 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.997006 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:20.997843 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.997823 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 18:55:20.997947 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.997905 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-spl7r\"" Jul 22 18:55:20.997947 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.997918 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 18:55:20.998664 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.998649 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:20.998773 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.998756 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 18:55:20.999230 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.999210 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:55:20.999369 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.999351 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 18:55:20.999616 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.999603 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 22 18:55:20.999696 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.999642 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-t24kc\"" Jul 22 18:55:20.999757 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:20.999712 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 22 18:55:21.003965 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.003947 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:55:21.005382 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.005367 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 22 18:55:21.005483 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.005402 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 22 18:55:21.005483 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.005420 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-j8c4c\"" Jul 22 18:55:21.005483 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.005408 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 22 18:55:21.012183 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.012163 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 22 18:55:21.025818 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.025796 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh"] Jul 22 18:55:21.026555 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.026535 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-lqrlz"] Jul 22 18:55:21.037749 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.037727 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm"] Jul 22 18:55:21.093255 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.093153 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c5889575d-xwd2v"] Jul 22 18:55:21.094971 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.094953 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.097820 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.097799 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 18:55:21.097904 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.097808 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 18:55:21.098060 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.098047 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d66st\"" Jul 22 18:55:21.103963 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.103945 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 18:55:21.107979 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.107960 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 18:55:21.112902 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.112884 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c5889575d-xwd2v"] Jul 22 18:55:21.130320 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.130299 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxmzl\" (UniqueName: \"kubernetes.io/projected/c8fd5e9a-db17-4529-9651-39ba7ed863fb-kube-api-access-mxmzl\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:21.130398 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.130335 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bc0db99a-e7b2-477a-adf5-2136fa37ac45-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:21.130398 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.130369 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7547bde7-d558-4a05-bdad-e06e98fdcb2a-serving-cert\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.130473 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.130419 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rkrm\" (UniqueName: \"kubernetes.io/projected/7547bde7-d558-4a05-bdad-e06e98fdcb2a-kube-api-access-5rkrm\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.130473 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.130453 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:21.130557 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.130471 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbhm4\" (UniqueName: \"kubernetes.io/projected/bc0db99a-e7b2-477a-adf5-2136fa37ac45-kube-api-access-kbhm4\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:21.130557 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.130495 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7547bde7-d558-4a05-bdad-e06e98fdcb2a-config\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.130557 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.130534 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:21.130654 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.130565 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7547bde7-d558-4a05-bdad-e06e98fdcb2a-trusted-ca\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.230984 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.230958 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rkrm\" (UniqueName: \"kubernetes.io/projected/7547bde7-d558-4a05-bdad-e06e98fdcb2a-kube-api-access-5rkrm\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.231131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.230993 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-certificates\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.231131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231013 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-trusted-ca\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.231131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231041 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.231131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231094 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:21.231131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231111 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbhm4\" (UniqueName: \"kubernetes.io/projected/bc0db99a-e7b2-477a-adf5-2136fa37ac45-kube-api-access-kbhm4\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:21.231131 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231126 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33072af6-3ef5-40e7-963a-7dc57d1df61c-ca-trust-extracted\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.231406 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231151 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7547bde7-d558-4a05-bdad-e06e98fdcb2a-config\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.231406 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231176 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:21.231406 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231194 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc98n\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-kube-api-access-lc98n\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.231406 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231212 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7547bde7-d558-4a05-bdad-e06e98fdcb2a-trusted-ca\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.231406 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.231218 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:21.231406 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231233 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-bound-sa-token\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.231406 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.231295 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls podName:bc0db99a-e7b2-477a-adf5-2136fa37ac45 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:21.731275293 +0000 UTC m=+255.393849624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mzswh" (UID: "bc0db99a-e7b2-477a-adf5-2136fa37ac45") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:21.231406 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.231309 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 18:55:21.231406 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231319 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxmzl\" (UniqueName: \"kubernetes.io/projected/c8fd5e9a-db17-4529-9651-39ba7ed863fb-kube-api-access-mxmzl\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:21.231863 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231352 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-image-registry-private-configuration\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.231863 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.231816 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls podName:c8fd5e9a-db17-4529-9651-39ba7ed863fb nodeName:}" failed. No retries permitted until 2026-07-22 18:55:21.731347224 +0000 UTC m=+255.393921555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-t8ltm" (UID: "c8fd5e9a-db17-4529-9651-39ba7ed863fb") : secret "samples-operator-tls" not found Jul 22 18:55:21.232037 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231884 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bc0db99a-e7b2-477a-adf5-2136fa37ac45-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:21.232137 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.231942 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7547bde7-d558-4a05-bdad-e06e98fdcb2a-serving-cert\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.232312 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.232288 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-installation-pull-secrets\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.232446 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.232424 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7547bde7-d558-4a05-bdad-e06e98fdcb2a-trusted-ca\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.232547 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.232351 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7547bde7-d558-4a05-bdad-e06e98fdcb2a-config\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.233948 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.233924 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/bc0db99a-e7b2-477a-adf5-2136fa37ac45-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:21.236533 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.236490 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7547bde7-d558-4a05-bdad-e06e98fdcb2a-serving-cert\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.240593 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.240559 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbhm4\" (UniqueName: \"kubernetes.io/projected/bc0db99a-e7b2-477a-adf5-2136fa37ac45-kube-api-access-kbhm4\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:21.241147 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.241123 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rkrm\" (UniqueName: \"kubernetes.io/projected/7547bde7-d558-4a05-bdad-e06e98fdcb2a-kube-api-access-5rkrm\") pod \"console-operator-565b7b569d-lqrlz\" (UID: \"7547bde7-d558-4a05-bdad-e06e98fdcb2a\") " pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.241610 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.241594 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxmzl\" (UniqueName: \"kubernetes.io/projected/c8fd5e9a-db17-4529-9651-39ba7ed863fb-kube-api-access-mxmzl\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:21.317270 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.317215 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:21.333714 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.333692 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-certificates\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.333813 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.333726 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-trusted-ca\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.333813 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.333756 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.333920 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.333871 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:55:21.333920 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.333886 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c5889575d-xwd2v: secret "image-registry-tls" not found Jul 22 18:55:21.333920 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.333886 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33072af6-3ef5-40e7-963a-7dc57d1df61c-ca-trust-extracted\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.334062 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.333937 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls podName:33072af6-3ef5-40e7-963a-7dc57d1df61c nodeName:}" failed. No retries permitted until 2026-07-22 18:55:21.833919963 +0000 UTC m=+255.496494302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls") pod "image-registry-c5889575d-xwd2v" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c") : secret "image-registry-tls" not found Jul 22 18:55:21.334062 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.333980 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc98n\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-kube-api-access-lc98n\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.334062 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.334012 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-bound-sa-token\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.334062 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.334056 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-image-registry-private-configuration\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.334279 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.334111 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-installation-pull-secrets\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.334279 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.334183 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33072af6-3ef5-40e7-963a-7dc57d1df61c-ca-trust-extracted\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.334279 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.334254 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-certificates\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.334631 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.334609 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-trusted-ca\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.336406 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.336387 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-installation-pull-secrets\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.336476 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.336431 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-image-registry-private-configuration\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.344483 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.344459 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-bound-sa-token\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.344646 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.344629 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc98n\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-kube-api-access-lc98n\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.435446 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.435347 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-lqrlz"] Jul 22 18:55:21.437823 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:55:21.437801 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7547bde7_d558_4a05_bdad_e06e98fdcb2a.slice/crio-f4bac52c2bf3a55f6c5401bb44d799aaca54386baec5dccacb1a45da81235de7 WatchSource:0}: Error finding container f4bac52c2bf3a55f6c5401bb44d799aaca54386baec5dccacb1a45da81235de7: Status 404 returned error can't find the container with id f4bac52c2bf3a55f6c5401bb44d799aaca54386baec5dccacb1a45da81235de7 Jul 22 18:55:21.582237 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.582207 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" event={"ID":"7547bde7-d558-4a05-bdad-e06e98fdcb2a","Type":"ContainerStarted","Data":"f4bac52c2bf3a55f6c5401bb44d799aaca54386baec5dccacb1a45da81235de7"} Jul 22 18:55:21.736834 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.736801 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:21.736834 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.736839 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:21.737030 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.736948 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:21.737030 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.737013 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 18:55:21.737114 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.737017 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls podName:bc0db99a-e7b2-477a-adf5-2136fa37ac45 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:22.737000895 +0000 UTC m=+256.399575214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mzswh" (UID: "bc0db99a-e7b2-477a-adf5-2136fa37ac45") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:21.737114 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.737059 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls podName:c8fd5e9a-db17-4529-9651-39ba7ed863fb nodeName:}" failed. No retries permitted until 2026-07-22 18:55:22.7370484 +0000 UTC m=+256.399622717 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-t8ltm" (UID: "c8fd5e9a-db17-4529-9651-39ba7ed863fb") : secret "samples-operator-tls" not found Jul 22 18:55:21.837894 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:21.837819 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:21.838014 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.837964 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:55:21.838014 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.837981 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c5889575d-xwd2v: secret "image-registry-tls" not found Jul 22 18:55:21.838090 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:21.838032 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls podName:33072af6-3ef5-40e7-963a-7dc57d1df61c nodeName:}" failed. No retries permitted until 2026-07-22 18:55:22.83801733 +0000 UTC m=+256.500591647 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls") pod "image-registry-c5889575d-xwd2v" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c") : secret "image-registry-tls" not found Jul 22 18:55:22.744771 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:22.744733 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:22.745180 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:22.744781 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:22.745180 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:22.744898 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 18:55:22.745180 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:22.744897 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:22.745180 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:22.744968 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls podName:bc0db99a-e7b2-477a-adf5-2136fa37ac45 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:24.744951295 +0000 UTC m=+258.407525612 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mzswh" (UID: "bc0db99a-e7b2-477a-adf5-2136fa37ac45") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:22.745180 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:22.744983 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls podName:c8fd5e9a-db17-4529-9651-39ba7ed863fb nodeName:}" failed. No retries permitted until 2026-07-22 18:55:24.744977239 +0000 UTC m=+258.407551556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-t8ltm" (UID: "c8fd5e9a-db17-4529-9651-39ba7ed863fb") : secret "samples-operator-tls" not found Jul 22 18:55:22.845822 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:22.845785 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:22.846010 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:22.845944 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:55:22.846010 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:22.845957 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c5889575d-xwd2v: secret "image-registry-tls" not found Jul 22 18:55:22.846132 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:22.846017 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls podName:33072af6-3ef5-40e7-963a-7dc57d1df61c nodeName:}" failed. No retries permitted until 2026-07-22 18:55:24.845999591 +0000 UTC m=+258.508573920 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls") pod "image-registry-c5889575d-xwd2v" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c") : secret "image-registry-tls" not found Jul 22 18:55:23.588205 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:23.588184 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/0.log" Jul 22 18:55:23.588363 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:23.588226 2584 generic.go:356] "Generic (PLEG): container finished" podID="7547bde7-d558-4a05-bdad-e06e98fdcb2a" containerID="f8e43aad1edeea6081af6a2e54c4d31f504c6ccf9732aa8835d2412898a14b27" exitCode=255 Jul 22 18:55:23.588363 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:23.588286 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" event={"ID":"7547bde7-d558-4a05-bdad-e06e98fdcb2a","Type":"ContainerDied","Data":"f8e43aad1edeea6081af6a2e54c4d31f504c6ccf9732aa8835d2412898a14b27"} Jul 22 18:55:23.588565 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:23.588548 2584 scope.go:117] "RemoveContainer" containerID="f8e43aad1edeea6081af6a2e54c4d31f504c6ccf9732aa8835d2412898a14b27" Jul 22 18:55:24.056223 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.056146 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kpw96"] Jul 22 18:55:24.058022 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.058005 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.060493 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.060471 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 18:55:24.060683 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.060665 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 18:55:24.060757 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.060746 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 18:55:24.061591 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.061578 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 18:55:24.061639 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.061590 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rcw4g\"" Jul 22 18:55:24.069638 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.069618 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kpw96"] Jul 22 18:55:24.156088 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.156050 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9bd8f044-f8df-42a4-abf9-59035972b0f4-data-volume\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.156225 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.156103 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.156225 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.156151 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j7gz\" (UniqueName: \"kubernetes.io/projected/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-api-access-7j7gz\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.156225 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.156194 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9bd8f044-f8df-42a4-abf9-59035972b0f4-crio-socket\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.156380 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.156275 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.256677 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.256641 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9bd8f044-f8df-42a4-abf9-59035972b0f4-data-volume\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.256677 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.256685 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.256854 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.256705 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j7gz\" (UniqueName: \"kubernetes.io/projected/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-api-access-7j7gz\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.256854 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.256730 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9bd8f044-f8df-42a4-abf9-59035972b0f4-crio-socket\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.256854 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.256789 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:55:24.256854 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.256800 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9bd8f044-f8df-42a4-abf9-59035972b0f4-crio-socket\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.256854 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.256851 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:24.756835838 +0000 UTC m=+258.419410154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : configmap "kube-rbac-proxy" not found Jul 22 18:55:24.257108 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.256904 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.257108 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.257001 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:55:24.257108 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.257036 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9bd8f044-f8df-42a4-abf9-59035972b0f4-data-volume\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.257108 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.257049 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:24.757038812 +0000 UTC m=+258.419613129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : secret "insights-runtime-extractor-tls" not found Jul 22 18:55:24.270040 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.270010 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j7gz\" (UniqueName: \"kubernetes.io/projected/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-api-access-7j7gz\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.591599 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.591573 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/1.log" Jul 22 18:55:24.591975 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.591958 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/0.log" Jul 22 18:55:24.592087 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.591999 2584 generic.go:356] "Generic (PLEG): container finished" podID="7547bde7-d558-4a05-bdad-e06e98fdcb2a" containerID="bcc794b253ddd782bb572fe0be569e738b76475f2e4cc5aa1a87a03abbda10b8" exitCode=255 Jul 22 18:55:24.592087 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.592060 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" event={"ID":"7547bde7-d558-4a05-bdad-e06e98fdcb2a","Type":"ContainerDied","Data":"bcc794b253ddd782bb572fe0be569e738b76475f2e4cc5aa1a87a03abbda10b8"} Jul 22 18:55:24.592182 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.592112 2584 scope.go:117] "RemoveContainer" containerID="f8e43aad1edeea6081af6a2e54c4d31f504c6ccf9732aa8835d2412898a14b27" Jul 22 18:55:24.592286 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.592270 2584 scope.go:117] "RemoveContainer" containerID="bcc794b253ddd782bb572fe0be569e738b76475f2e4cc5aa1a87a03abbda10b8" Jul 22 18:55:24.592427 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.592408 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-lqrlz_openshift-console-operator(7547bde7-d558-4a05-bdad-e06e98fdcb2a)\"" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" podUID="7547bde7-d558-4a05-bdad-e06e98fdcb2a" Jul 22 18:55:24.760658 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.760632 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:24.760761 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.760663 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:24.760761 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.760681 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.760761 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.760747 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:24.760893 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.760762 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:24.760893 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.760780 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 18:55:24.760893 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.760814 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls podName:bc0db99a-e7b2-477a-adf5-2136fa37ac45 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:28.760801228 +0000 UTC m=+262.423375544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mzswh" (UID: "bc0db99a-e7b2-477a-adf5-2136fa37ac45") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:24.760893 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.760832 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls podName:c8fd5e9a-db17-4529-9651-39ba7ed863fb nodeName:}" failed. No retries permitted until 2026-07-22 18:55:28.760820739 +0000 UTC m=+262.423395056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-t8ltm" (UID: "c8fd5e9a-db17-4529-9651-39ba7ed863fb") : secret "samples-operator-tls" not found Jul 22 18:55:24.760893 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.760834 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:55:24.760893 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.760855 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:55:24.760893 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.760877 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:25.760864259 +0000 UTC m=+259.423438587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : configmap "kube-rbac-proxy" not found Jul 22 18:55:24.760893 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.760893 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:25.760882289 +0000 UTC m=+259.423456613 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : secret "insights-runtime-extractor-tls" not found Jul 22 18:55:24.861132 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:24.861065 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:24.861242 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.861199 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:55:24.861242 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.861214 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c5889575d-xwd2v: secret "image-registry-tls" not found Jul 22 18:55:24.861355 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:24.861261 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls podName:33072af6-3ef5-40e7-963a-7dc57d1df61c nodeName:}" failed. No retries permitted until 2026-07-22 18:55:28.861246096 +0000 UTC m=+262.523820416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls") pod "image-registry-c5889575d-xwd2v" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c") : secret "image-registry-tls" not found Jul 22 18:55:25.595685 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:25.595657 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/1.log" Jul 22 18:55:25.596055 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:25.595970 2584 scope.go:117] "RemoveContainer" containerID="bcc794b253ddd782bb572fe0be569e738b76475f2e4cc5aa1a87a03abbda10b8" Jul 22 18:55:25.596126 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:25.596107 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-lqrlz_openshift-console-operator(7547bde7-d558-4a05-bdad-e06e98fdcb2a)\"" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" podUID="7547bde7-d558-4a05-bdad-e06e98fdcb2a" Jul 22 18:55:25.769946 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:25.769883 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:25.770150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:25.770001 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:25.770150 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:25.770026 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:55:25.770150 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:25.770094 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:27.770078369 +0000 UTC m=+261.432652686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : configmap "kube-rbac-proxy" not found Jul 22 18:55:25.770150 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:25.770141 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:55:25.770337 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:25.770204 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:27.770187262 +0000 UTC m=+261.432761584 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : secret "insights-runtime-extractor-tls" not found Jul 22 18:55:26.220491 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:26.220460 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lpxbp_62f16011-c0a3-42ac-ace9-fa028fc57f76/dns-node-resolver/0.log" Jul 22 18:55:27.223145 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.223073 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g2xmq_5c280d2a-6833-4775-a794-317b88a86303/node-ca/0.log" Jul 22 18:55:27.783631 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.783584 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:27.783815 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.783656 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:27.783815 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:27.783706 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:55:27.783815 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:27.783742 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:55:27.783815 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:27.783769 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:31.783753076 +0000 UTC m=+265.446327393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : configmap "kube-rbac-proxy" not found Jul 22 18:55:27.783815 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:27.783783 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:31.783776747 +0000 UTC m=+265.446351064 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : secret "insights-runtime-extractor-tls" not found Jul 22 18:55:27.797274 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.797241 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bfmd4"] Jul 22 18:55:27.799636 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.799615 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:27.802208 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.802186 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 18:55:27.802316 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.802186 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 18:55:27.802316 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.802298 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 18:55:27.803402 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.803383 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lctpf\"" Jul 22 18:55:27.803495 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.803436 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 18:55:27.808048 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.808024 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bfmd4"] Jul 22 18:55:27.821137 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.821118 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7qjbt_5037332a-7cf0-489d-b14c-ce356a5a3c4d/serve-healthcheck-canary/0.log" Jul 22 18:55:27.884442 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.884416 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91cfec15-6fad-409a-9b27-e6ecad645d37-signing-key\") pod \"service-ca-58598695f7-bfmd4\" (UID: \"91cfec15-6fad-409a-9b27-e6ecad645d37\") " pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:27.884573 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.884468 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdg9j\" (UniqueName: \"kubernetes.io/projected/91cfec15-6fad-409a-9b27-e6ecad645d37-kube-api-access-kdg9j\") pod \"service-ca-58598695f7-bfmd4\" (UID: \"91cfec15-6fad-409a-9b27-e6ecad645d37\") " pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:27.884573 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.884557 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91cfec15-6fad-409a-9b27-e6ecad645d37-signing-cabundle\") pod \"service-ca-58598695f7-bfmd4\" (UID: \"91cfec15-6fad-409a-9b27-e6ecad645d37\") " pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:27.984959 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.984928 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdg9j\" (UniqueName: \"kubernetes.io/projected/91cfec15-6fad-409a-9b27-e6ecad645d37-kube-api-access-kdg9j\") pod \"service-ca-58598695f7-bfmd4\" (UID: \"91cfec15-6fad-409a-9b27-e6ecad645d37\") " pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:27.985093 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.984979 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91cfec15-6fad-409a-9b27-e6ecad645d37-signing-cabundle\") pod \"service-ca-58598695f7-bfmd4\" (UID: \"91cfec15-6fad-409a-9b27-e6ecad645d37\") " pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:27.985093 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.985051 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91cfec15-6fad-409a-9b27-e6ecad645d37-signing-key\") pod \"service-ca-58598695f7-bfmd4\" (UID: \"91cfec15-6fad-409a-9b27-e6ecad645d37\") " pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:27.985774 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.985755 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91cfec15-6fad-409a-9b27-e6ecad645d37-signing-cabundle\") pod \"service-ca-58598695f7-bfmd4\" (UID: \"91cfec15-6fad-409a-9b27-e6ecad645d37\") " pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:27.987580 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.987563 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91cfec15-6fad-409a-9b27-e6ecad645d37-signing-key\") pod \"service-ca-58598695f7-bfmd4\" (UID: \"91cfec15-6fad-409a-9b27-e6ecad645d37\") " pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:27.993031 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:27.993010 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdg9j\" (UniqueName: \"kubernetes.io/projected/91cfec15-6fad-409a-9b27-e6ecad645d37-kube-api-access-kdg9j\") pod \"service-ca-58598695f7-bfmd4\" (UID: \"91cfec15-6fad-409a-9b27-e6ecad645d37\") " pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:28.109944 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:28.109918 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-bfmd4" Jul 22 18:55:28.227597 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:28.227564 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bfmd4"] Jul 22 18:55:28.230107 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:55:28.230083 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91cfec15_6fad_409a_9b27_e6ecad645d37.slice/crio-702f5d885291ff3de0e62bf44433cc2eb2ad359a245de955ccfc574ae64c2bb9 WatchSource:0}: Error finding container 702f5d885291ff3de0e62bf44433cc2eb2ad359a245de955ccfc574ae64c2bb9: Status 404 returned error can't find the container with id 702f5d885291ff3de0e62bf44433cc2eb2ad359a245de955ccfc574ae64c2bb9 Jul 22 18:55:28.603638 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:28.603606 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-bfmd4" event={"ID":"91cfec15-6fad-409a-9b27-e6ecad645d37","Type":"ContainerStarted","Data":"702f5d885291ff3de0e62bf44433cc2eb2ad359a245de955ccfc574ae64c2bb9"} Jul 22 18:55:28.791749 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:28.791718 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:28.791871 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:28.791754 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:28.791932 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:28.791876 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:28.791968 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:28.791932 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls podName:bc0db99a-e7b2-477a-adf5-2136fa37ac45 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:36.791917363 +0000 UTC m=+270.454491680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mzswh" (UID: "bc0db99a-e7b2-477a-adf5-2136fa37ac45") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:28.791968 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:28.791876 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 18:55:28.792032 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:28.791993 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls podName:c8fd5e9a-db17-4529-9651-39ba7ed863fb nodeName:}" failed. No retries permitted until 2026-07-22 18:55:36.791981985 +0000 UTC m=+270.454556304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-t8ltm" (UID: "c8fd5e9a-db17-4529-9651-39ba7ed863fb") : secret "samples-operator-tls" not found Jul 22 18:55:28.892886 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:28.892816 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:28.893037 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:28.892975 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:55:28.893037 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:28.892991 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c5889575d-xwd2v: secret "image-registry-tls" not found Jul 22 18:55:28.893143 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:28.893046 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls podName:33072af6-3ef5-40e7-963a-7dc57d1df61c nodeName:}" failed. No retries permitted until 2026-07-22 18:55:36.893030355 +0000 UTC m=+270.555604691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls") pod "image-registry-c5889575d-xwd2v" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c") : secret "image-registry-tls" not found Jul 22 18:55:30.610584 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:30.610547 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-bfmd4" event={"ID":"91cfec15-6fad-409a-9b27-e6ecad645d37","Type":"ContainerStarted","Data":"6815bc339d2fb90396edfce9cf46cc2719161c9ba9d61cdf82db14fff03602e6"} Jul 22 18:55:30.631876 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:30.629502 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-bfmd4" podStartSLOduration=1.673880069 podStartE2EDuration="3.629482401s" totalImagesPullingTime="1.955602332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:55:30.626846926 +0000 UTC m=+264.289421265" watchObservedRunningTime="2026-07-22 18:55:30.629482401 +0000 UTC m=+264.292056743" Jul 22 18:55:31.318402 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:31.318319 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:31.318402 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:31.318368 2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:31.318702 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:31.318684 2584 scope.go:117] "RemoveContainer" containerID="bcc794b253ddd782bb572fe0be569e738b76475f2e4cc5aa1a87a03abbda10b8" Jul 22 18:55:31.318846 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:31.318824 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-lqrlz_openshift-console-operator(7547bde7-d558-4a05-bdad-e06e98fdcb2a)\"" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" podUID="7547bde7-d558-4a05-bdad-e06e98fdcb2a" Jul 22 18:55:31.816826 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:31.816793 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:31.817226 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:31.816882 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:31.817226 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:31.816914 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:55:31.817226 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:31.816981 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:39.816958502 +0000 UTC m=+273.479532821 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : configmap "kube-rbac-proxy" not found Jul 22 18:55:31.817226 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:31.817035 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:55:31.817226 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:31.817087 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:39.817075164 +0000 UTC m=+273.479649484 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : secret "insights-runtime-extractor-tls" not found Jul 22 18:55:36.857365 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:36.857333 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:36.857365 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:36.857368 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:36.857795 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:36.857479 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:36.857795 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:36.857564 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls podName:bc0db99a-e7b2-477a-adf5-2136fa37ac45 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:52.857544824 +0000 UTC m=+286.520119140 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mzswh" (UID: "bc0db99a-e7b2-477a-adf5-2136fa37ac45") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:55:36.859995 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:36.859973 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c8fd5e9a-db17-4529-9651-39ba7ed863fb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-t8ltm\" (UID: \"c8fd5e9a-db17-4529-9651-39ba7ed863fb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:36.911366 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:36.911343 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" Jul 22 18:55:36.958104 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:36.958069 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:36.960831 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:36.960807 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls\") pod \"image-registry-c5889575d-xwd2v\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:37.004174 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:37.004147 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:37.033891 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:37.033817 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm"] Jul 22 18:55:37.138385 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:37.138317 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c5889575d-xwd2v"] Jul 22 18:55:37.140815 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:55:37.140793 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33072af6_3ef5_40e7_963a_7dc57d1df61c.slice/crio-dd9e8f46c2f73dc9760e92aa7229173d9665e42ebc5efbb2ae8101b190eef36b WatchSource:0}: Error finding container dd9e8f46c2f73dc9760e92aa7229173d9665e42ebc5efbb2ae8101b190eef36b: Status 404 returned error can't find the container with id dd9e8f46c2f73dc9760e92aa7229173d9665e42ebc5efbb2ae8101b190eef36b Jul 22 18:55:37.628229 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:37.628190 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" event={"ID":"c8fd5e9a-db17-4529-9651-39ba7ed863fb","Type":"ContainerStarted","Data":"9c717c6cfe64def124628c531a70136c489716c0a1993ef540faf719bd53aa1a"} Jul 22 18:55:37.629572 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:37.629548 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" event={"ID":"33072af6-3ef5-40e7-963a-7dc57d1df61c","Type":"ContainerStarted","Data":"3d180acb6f23bd281e57061957ec27187a1b743c0700d36cc09e0030737972d2"} Jul 22 18:55:37.629618 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:37.629595 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" event={"ID":"33072af6-3ef5-40e7-963a-7dc57d1df61c","Type":"ContainerStarted","Data":"dd9e8f46c2f73dc9760e92aa7229173d9665e42ebc5efbb2ae8101b190eef36b"} Jul 22 18:55:37.629721 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:37.629703 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:55:37.650877 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:37.650836 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" podStartSLOduration=16.650823292 podStartE2EDuration="16.650823292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:55:37.649177918 +0000 UTC m=+271.311752259" watchObservedRunningTime="2026-07-22 18:55:37.650823292 +0000 UTC m=+271.313397630" Jul 22 18:55:39.637234 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:39.637196 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" event={"ID":"c8fd5e9a-db17-4529-9651-39ba7ed863fb","Type":"ContainerStarted","Data":"acc81dd6424c13afe75c7749ff088a37eda3dac99f28e203f66028e1c6752731"} Jul 22 18:55:39.637593 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:39.637261 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" event={"ID":"c8fd5e9a-db17-4529-9651-39ba7ed863fb","Type":"ContainerStarted","Data":"fccab3e74b653ee6f65cc2241462a60fa85b8cdd14f4bf832d70867c4db4f102"} Jul 22 18:55:39.653713 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:39.653673 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-t8ltm" podStartSLOduration=17.571114611 podStartE2EDuration="19.653651564s" totalImagesPullingTime="2.082536953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:55:39.653058957 +0000 UTC m=+273.315633296" watchObservedRunningTime="2026-07-22 18:55:39.653651564 +0000 UTC m=+273.316225903" Jul 22 18:55:39.885754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:39.885716 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:39.885897 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:39.885773 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:39.885897 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:39.885878 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:55:39.885992 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:39.885929 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls podName:9bd8f044-f8df-42a4-abf9-59035972b0f4 nodeName:}" failed. No retries permitted until 2026-07-22 18:55:55.885916669 +0000 UTC m=+289.548490986 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kpw96" (UID: "9bd8f044-f8df-42a4-abf9-59035972b0f4") : secret "insights-runtime-extractor-tls" not found Jul 22 18:55:39.886268 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:39.886248 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9bd8f044-f8df-42a4-abf9-59035972b0f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:46.901775 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:46.901748 2584 scope.go:117] "RemoveContainer" containerID="bcc794b253ddd782bb572fe0be569e738b76475f2e4cc5aa1a87a03abbda10b8" Jul 22 18:55:47.661727 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:47.661701 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 18:55:47.662096 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:47.662082 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/1.log" Jul 22 18:55:47.662153 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:47.662113 2584 generic.go:356] "Generic (PLEG): container finished" podID="7547bde7-d558-4a05-bdad-e06e98fdcb2a" containerID="f054cad89f71f3c8d17c83653439d44610fb7efe78017b5b6f412508387a7d5b" exitCode=255 Jul 22 18:55:47.662153 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:47.662144 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" event={"ID":"7547bde7-d558-4a05-bdad-e06e98fdcb2a","Type":"ContainerDied","Data":"f054cad89f71f3c8d17c83653439d44610fb7efe78017b5b6f412508387a7d5b"} Jul 22 18:55:47.662224 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:47.662213 2584 scope.go:117] "RemoveContainer" containerID="bcc794b253ddd782bb572fe0be569e738b76475f2e4cc5aa1a87a03abbda10b8" Jul 22 18:55:47.662509 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:47.662485 2584 scope.go:117] "RemoveContainer" containerID="f054cad89f71f3c8d17c83653439d44610fb7efe78017b5b6f412508387a7d5b" Jul 22 18:55:47.662712 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:47.662692 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-lqrlz_openshift-console-operator(7547bde7-d558-4a05-bdad-e06e98fdcb2a)\"" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" podUID="7547bde7-d558-4a05-bdad-e06e98fdcb2a" Jul 22 18:55:48.363498 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:48.363459 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7lk8l" podUID="b6144392-e95f-4cf4-973c-f73ce61a0780" Jul 22 18:55:48.539155 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:48.539126 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c5889575d-xwd2v"] Jul 22 18:55:48.666287 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:48.666211 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 18:55:48.666418 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:48.666294 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lk8l" Jul 22 18:55:51.317882 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:51.317833 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:51.318392 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:51.317906 2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:55:51.318392 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:51.318210 2584 scope.go:117] "RemoveContainer" containerID="f054cad89f71f3c8d17c83653439d44610fb7efe78017b5b6f412508387a7d5b" Jul 22 18:55:51.318392 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:55:51.318359 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-lqrlz_openshift-console-operator(7547bde7-d558-4a05-bdad-e06e98fdcb2a)\"" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" podUID="7547bde7-d558-4a05-bdad-e06e98fdcb2a" Jul 22 18:55:52.079283 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:52.079237 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:55:52.081742 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:52.081720 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b6144392-e95f-4cf4-973c-f73ce61a0780-metrics-tls\") pod \"dns-default-7lk8l\" (UID: \"b6144392-e95f-4cf4-973c-f73ce61a0780\") " pod="openshift-dns/dns-default-7lk8l" Jul 22 18:55:52.272178 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:52.272147 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-phz5q\"" Jul 22 18:55:52.278232 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:52.278213 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lk8l" Jul 22 18:55:52.402490 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:52.402422 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7lk8l"] Jul 22 18:55:52.404964 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:55:52.404938 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6144392_e95f_4cf4_973c_f73ce61a0780.slice/crio-13bd175a396299e8a8d1e5e2103ddbaa0f3d6d71745a0d12aa8a752626d23be3 WatchSource:0}: Error finding container 13bd175a396299e8a8d1e5e2103ddbaa0f3d6d71745a0d12aa8a752626d23be3: Status 404 returned error can't find the container with id 13bd175a396299e8a8d1e5e2103ddbaa0f3d6d71745a0d12aa8a752626d23be3 Jul 22 18:55:52.678500 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:52.678413 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lk8l" event={"ID":"b6144392-e95f-4cf4-973c-f73ce61a0780","Type":"ContainerStarted","Data":"13bd175a396299e8a8d1e5e2103ddbaa0f3d6d71745a0d12aa8a752626d23be3"} Jul 22 18:55:52.885099 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:52.885043 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:52.888062 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:52.888036 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/bc0db99a-e7b2-477a-adf5-2136fa37ac45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mzswh\" (UID: \"bc0db99a-e7b2-477a-adf5-2136fa37ac45\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:53.105415 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:53.105377 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" Jul 22 18:55:53.268918 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:53.268838 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh"] Jul 22 18:55:53.272948 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:55:53.272912 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc0db99a_e7b2_477a_adf5_2136fa37ac45.slice/crio-3cbcbdace18629e78f4743fe664f5ba67de230b7ab809dbf59ef4236bf1ced24 WatchSource:0}: Error finding container 3cbcbdace18629e78f4743fe664f5ba67de230b7ab809dbf59ef4236bf1ced24: Status 404 returned error can't find the container with id 3cbcbdace18629e78f4743fe664f5ba67de230b7ab809dbf59ef4236bf1ced24 Jul 22 18:55:53.683267 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:53.683231 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" event={"ID":"bc0db99a-e7b2-477a-adf5-2136fa37ac45","Type":"ContainerStarted","Data":"3cbcbdace18629e78f4743fe664f5ba67de230b7ab809dbf59ef4236bf1ced24"} Jul 22 18:55:54.689090 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:54.689053 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lk8l" event={"ID":"b6144392-e95f-4cf4-973c-f73ce61a0780","Type":"ContainerStarted","Data":"7a9ab05a579744c3b9aa38000a8ca6e66855a492334af743226633720129daf5"} Jul 22 18:55:54.689583 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:54.689111 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lk8l" event={"ID":"b6144392-e95f-4cf4-973c-f73ce61a0780","Type":"ContainerStarted","Data":"a2cad06504fa93eb61e1eac619432e16bb8c3904a7ec804c7a7259dec2202ad3"} Jul 22 18:55:54.689583 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:54.689201 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7lk8l" Jul 22 18:55:54.705842 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:54.705793 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7lk8l" podStartSLOduration=251.200690975 podStartE2EDuration="4m12.705778685s" totalImagesPullingTime="1.50508771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:55:54.705241936 +0000 UTC m=+288.367816276" watchObservedRunningTime="2026-07-22 18:55:54.705778685 +0000 UTC m=+288.368353026" Jul 22 18:55:55.694003 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:55.693961 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" event={"ID":"bc0db99a-e7b2-477a-adf5-2136fa37ac45","Type":"ContainerStarted","Data":"d03b713aa893ba0eeed88b71af22b1c25905206bf86454385100f800b8256800"} Jul 22 18:55:55.711505 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:55.711457 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mzswh" podStartSLOduration=33.788035842 podStartE2EDuration="35.711439979s" totalImagesPullingTime="1.923404137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:55:55.709997084 +0000 UTC m=+289.372571424" watchObservedRunningTime="2026-07-22 18:55:55.711439979 +0000 UTC m=+289.374014309" Jul 22 18:55:55.907378 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:55.907340 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:55.909724 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:55.909701 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9bd8f044-f8df-42a4-abf9-59035972b0f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kpw96\" (UID: \"9bd8f044-f8df-42a4-abf9-59035972b0f4\") " pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:56.167022 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:56.166985 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kpw96" Jul 22 18:55:56.290901 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:56.290835 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kpw96"] Jul 22 18:55:56.293063 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:55:56.293043 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bd8f044_f8df_42a4_abf9_59035972b0f4.slice/crio-a53905c656e965fbcf9ccbaea8c048a10e698974fb1ebc9c6e952ba89f3139aa WatchSource:0}: Error finding container a53905c656e965fbcf9ccbaea8c048a10e698974fb1ebc9c6e952ba89f3139aa: Status 404 returned error can't find the container with id a53905c656e965fbcf9ccbaea8c048a10e698974fb1ebc9c6e952ba89f3139aa Jul 22 18:55:56.699535 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:56.699486 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kpw96" event={"ID":"9bd8f044-f8df-42a4-abf9-59035972b0f4","Type":"ContainerStarted","Data":"a53905c656e965fbcf9ccbaea8c048a10e698974fb1ebc9c6e952ba89f3139aa"} Jul 22 18:55:58.545619 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:55:58.545593 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:56:01.717847 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:01.717810 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kpw96" event={"ID":"9bd8f044-f8df-42a4-abf9-59035972b0f4","Type":"ContainerStarted","Data":"c7f8d0ed9047684f351991e169ed69ebf65996b854e1198975e255828a45ae29"} Jul 22 18:56:04.696941 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:04.696906 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7lk8l" Jul 22 18:56:04.901187 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:56:04.901151 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-l98h2" podUID="d6610a82-7a15-4b2f-9d80-8d2eae96e69e" Jul 22 18:56:05.121536 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.121488 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-6lcng"] Jul 22 18:56:05.123660 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.123639 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.127109 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.127019 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 22 18:56:05.127109 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.127031 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 22 18:56:05.127109 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.127053 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 18:56:05.127302 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.127021 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 22 18:56:05.127302 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.127147 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-j96q8\"" Jul 22 18:56:05.132345 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.132323 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d96jc"] Jul 22 18:56:05.134253 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.134236 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.137738 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.137713 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 18:56:05.137837 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.137802 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 18:56:05.137837 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.137824 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gzn74\"" Jul 22 18:56:05.137938 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.137918 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 18:56:05.138499 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.138483 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-6lcng"] Jul 22 18:56:05.189279 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189247 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.189490 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189304 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr2jx\" (UniqueName: \"kubernetes.io/projected/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-api-access-zr2jx\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.189490 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189332 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-textfile\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.189490 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189351 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.189490 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189370 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.189490 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189389 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df232a8d-6dd2-495e-857a-15a38b6d84a8-metrics-client-ca\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.189490 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189453 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df232a8d-6dd2-495e-857a-15a38b6d84a8-sys\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.189824 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189526 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/df232a8d-6dd2-495e-857a-15a38b6d84a8-root\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.189824 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189548 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.189824 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189575 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.189824 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189599 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.189824 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189618 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-wtmp\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.189824 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189633 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-tls\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.189824 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189663 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.189824 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.189690 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm87f\" (UniqueName: \"kubernetes.io/projected/df232a8d-6dd2-495e-857a-15a38b6d84a8-kube-api-access-lm87f\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.290694 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290662 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.290843 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290715 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr2jx\" (UniqueName: \"kubernetes.io/projected/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-api-access-zr2jx\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.290843 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290744 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-textfile\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.290843 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290769 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.290843 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290795 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.290843 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290823 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df232a8d-6dd2-495e-857a-15a38b6d84a8-metrics-client-ca\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.291111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290850 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df232a8d-6dd2-495e-857a-15a38b6d84a8-sys\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.291111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290877 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/df232a8d-6dd2-495e-857a-15a38b6d84a8-root\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.291111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290900 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.291111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290928 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.291111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290934 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df232a8d-6dd2-495e-857a-15a38b6d84a8-sys\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.291111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.290966 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.291111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.291001 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-wtmp\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.291111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.291025 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-tls\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.291111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.291075 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.291111 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.291106 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm87f\" (UniqueName: \"kubernetes.io/projected/df232a8d-6dd2-495e-857a-15a38b6d84a8-kube-api-access-lm87f\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.291580 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.291192 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-textfile\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.291580 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.291275 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.291580 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:56:05.291295 2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 22 18:56:05.291580 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.291333 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/df232a8d-6dd2-495e-857a-15a38b6d84a8-root\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.291580 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:56:05.291351 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-tls podName:df232a8d-6dd2-495e-857a-15a38b6d84a8 nodeName:}" failed. No retries permitted until 2026-07-22 18:56:05.791331808 +0000 UTC m=+299.453906131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-tls") pod "node-exporter-d96jc" (UID: "df232a8d-6dd2-495e-857a-15a38b6d84a8") : secret "node-exporter-tls" not found Jul 22 18:56:05.291818 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.291604 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.291818 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.291729 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-wtmp\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.292183 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.292156 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df232a8d-6dd2-495e-857a-15a38b6d84a8-metrics-client-ca\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.292364 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.292343 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.292618 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.292596 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.294259 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.294231 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.294373 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.294288 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.294434 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.294389 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.300927 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.300907 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm87f\" (UniqueName: \"kubernetes.io/projected/df232a8d-6dd2-495e-857a-15a38b6d84a8-kube-api-access-lm87f\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.301287 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.301266 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr2jx\" (UniqueName: \"kubernetes.io/projected/4b6a3e3f-cc74-414a-9c3f-964f79b650d3-kube-api-access-zr2jx\") pod \"kube-state-metrics-56c5594674-6lcng\" (UID: \"4b6a3e3f-cc74-414a-9c3f-964f79b650d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.434449 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.434378 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" Jul 22 18:56:05.623065 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.622942 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-6lcng"] Jul 22 18:56:05.626533 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:56:05.626486 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6a3e3f_cc74_414a_9c3f_964f79b650d3.slice/crio-d70efd5da16194f1e55811d2e9e03c51afdaa2c1762438875fe75d1b10061b35 WatchSource:0}: Error finding container d70efd5da16194f1e55811d2e9e03c51afdaa2c1762438875fe75d1b10061b35: Status 404 returned error can't find the container with id d70efd5da16194f1e55811d2e9e03c51afdaa2c1762438875fe75d1b10061b35 Jul 22 18:56:05.734034 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.733951 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" event={"ID":"4b6a3e3f-cc74-414a-9c3f-964f79b650d3","Type":"ContainerStarted","Data":"d70efd5da16194f1e55811d2e9e03c51afdaa2c1762438875fe75d1b10061b35"} Jul 22 18:56:05.736067 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.736020 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kpw96" event={"ID":"9bd8f044-f8df-42a4-abf9-59035972b0f4","Type":"ContainerStarted","Data":"615380d168d6b8de871c0efe7403f0544498f5cb37d972a210c954f79074ff77"} Jul 22 18:56:05.795754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.795718 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-tls\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.798663 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.798637 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df232a8d-6dd2-495e-857a-15a38b6d84a8-node-exporter-tls\") pod \"node-exporter-d96jc\" (UID: \"df232a8d-6dd2-495e-857a-15a38b6d84a8\") " pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:05.900912 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:05.900839 2584 scope.go:117] "RemoveContainer" containerID="f054cad89f71f3c8d17c83653439d44610fb7efe78017b5b6f412508387a7d5b" Jul 22 18:56:05.901082 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:56:05.901050 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-lqrlz_openshift-console-operator(7547bde7-d558-4a05-bdad-e06e98fdcb2a)\"" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" podUID="7547bde7-d558-4a05-bdad-e06e98fdcb2a" Jul 22 18:56:06.044440 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:06.044348 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d96jc" Jul 22 18:56:06.071380 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:56:06.070908 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf232a8d_6dd2_495e_857a_15a38b6d84a8.slice/crio-d287f92af079c42328610ab3a0fcfe1045eb9f59ac08f0e5fe7df9289b98a603 WatchSource:0}: Error finding container d287f92af079c42328610ab3a0fcfe1045eb9f59ac08f0e5fe7df9289b98a603: Status 404 returned error can't find the container with id d287f92af079c42328610ab3a0fcfe1045eb9f59ac08f0e5fe7df9289b98a603 Jul 22 18:56:06.745351 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:06.745279 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d96jc" event={"ID":"df232a8d-6dd2-495e-857a-15a38b6d84a8","Type":"ContainerStarted","Data":"d287f92af079c42328610ab3a0fcfe1045eb9f59ac08f0e5fe7df9289b98a603"} Jul 22 18:56:09.102982 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.102560 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 18:56:09.102982 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.102785 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 18:56:09.113433 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.113413 2584 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 18:56:09.610779 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.610694 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8bd746b97-mxgnr"] Jul 22 18:56:09.613911 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.613888 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.617904 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.617880 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cmb67on9udct5\"" Jul 22 18:56:09.618024 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.617889 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 22 18:56:09.618024 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.617889 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 22 18:56:09.618024 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.617889 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 22 18:56:09.618024 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.617990 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7vbbl\"" Jul 22 18:56:09.618226 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.617889 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 22 18:56:09.623730 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.623709 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8bd746b97-mxgnr"] Jul 22 18:56:09.731048 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.731010 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-secret-metrics-server-client-certs\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.731183 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.731068 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtnqm\" (UniqueName: \"kubernetes.io/projected/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-kube-api-access-wtnqm\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.731183 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.731113 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.731183 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.731157 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-client-ca-bundle\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.731301 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.731185 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-audit-log\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.731301 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.731278 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-secret-metrics-server-tls\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.731364 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.731330 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-metrics-server-audit-profiles\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.758880 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.758850 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" event={"ID":"4b6a3e3f-cc74-414a-9c3f-964f79b650d3","Type":"ContainerStarted","Data":"2f1caaf19f5a4884c27f4e7845033f482c6dc5c2922a3a843580b7d4990ff128"} Jul 22 18:56:09.759020 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.758918 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" event={"ID":"4b6a3e3f-cc74-414a-9c3f-964f79b650d3","Type":"ContainerStarted","Data":"1d1cd586c263490176fa934cbacc8136c275c1e6bca7190dcf06870234e1b664"} Jul 22 18:56:09.759020 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.758944 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" event={"ID":"4b6a3e3f-cc74-414a-9c3f-964f79b650d3","Type":"ContainerStarted","Data":"846a0fd2db466df20db9fd81fe19603a36a38bde5a9eb1271ecc0f09b2737326"} Jul 22 18:56:09.760302 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.760280 2584 generic.go:356] "Generic (PLEG): container finished" podID="df232a8d-6dd2-495e-857a-15a38b6d84a8" containerID="1a5d087058acb27475995f2371082b8ba81f1602b2852b0d827a32705965a515" exitCode=0 Jul 22 18:56:09.760404 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.760360 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d96jc" event={"ID":"df232a8d-6dd2-495e-857a-15a38b6d84a8","Type":"ContainerDied","Data":"1a5d087058acb27475995f2371082b8ba81f1602b2852b0d827a32705965a515"} Jul 22 18:56:09.762361 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.762341 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kpw96" event={"ID":"9bd8f044-f8df-42a4-abf9-59035972b0f4","Type":"ContainerStarted","Data":"cc50c2d74ce6414719bb0ceb4bd96996e991c54ed29da436640cf25709180059"} Jul 22 18:56:09.778731 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.778691 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-6lcng" podStartSLOduration=1.317604158 podStartE2EDuration="4.778677731s" totalImagesPullingTime="3.461073573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:09.777573094 +0000 UTC m=+303.440147435" watchObservedRunningTime="2026-07-22 18:56:09.778677731 +0000 UTC m=+303.441252070" Jul 22 18:56:09.815108 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.815056 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kpw96" podStartSLOduration=33.119652631 podStartE2EDuration="45.81503979s" totalImagesPullingTime="12.695387159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:55:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:09.813941229 +0000 UTC m=+303.476515569" watchObservedRunningTime="2026-07-22 18:56:09.81503979 +0000 UTC m=+303.477614128" Jul 22 18:56:09.831978 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.831946 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtnqm\" (UniqueName: \"kubernetes.io/projected/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-kube-api-access-wtnqm\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.832124 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.832019 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.832276 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.832253 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-client-ca-bundle\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.832358 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.832314 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-audit-log\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.832420 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.832398 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-secret-metrics-server-tls\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.832475 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.832441 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-metrics-server-audit-profiles\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.832688 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.832668 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-secret-metrics-server-client-certs\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.832789 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.832712 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.833587 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.833485 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-audit-log\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.833708 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.833601 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-metrics-server-audit-profiles\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.836785 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.836753 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-secret-metrics-server-tls\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.837150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.837048 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-secret-metrics-server-client-certs\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.838246 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.838217 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-client-ca-bundle\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.842597 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.842575 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtnqm\" (UniqueName: \"kubernetes.io/projected/70614419-09b5-4fea-9d3b-aa9bccf3c6c5-kube-api-access-wtnqm\") pod \"metrics-server-8bd746b97-mxgnr\" (UID: \"70614419-09b5-4fea-9d3b-aa9bccf3c6c5\") " pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:09.924000 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:09.923975 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:10.053178 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:10.053143 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8bd746b97-mxgnr"] Jul 22 18:56:10.055784 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:56:10.055757 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70614419_09b5_4fea_9d3b_aa9bccf3c6c5.slice/crio-f2c05b80f8a2d7267d6c4c3e1105184ea8a952119d3942311a5198f7207cc287 WatchSource:0}: Error finding container f2c05b80f8a2d7267d6c4c3e1105184ea8a952119d3942311a5198f7207cc287: Status 404 returned error can't find the container with id f2c05b80f8a2d7267d6c4c3e1105184ea8a952119d3942311a5198f7207cc287 Jul 22 18:56:10.057630 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:10.057613 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 18:56:10.769307 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:10.769265 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d96jc" event={"ID":"df232a8d-6dd2-495e-857a-15a38b6d84a8","Type":"ContainerStarted","Data":"a9f0e786bcc7298b770b26e33627c89fde7f3739221fcc594bb159c1d7167978"} Jul 22 18:56:10.769793 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:10.769380 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d96jc" event={"ID":"df232a8d-6dd2-495e-857a-15a38b6d84a8","Type":"ContainerStarted","Data":"cc3cf101d0659efff68ecd2f07b481d2442e9fd53035127e087db15ad52a4d48"} Jul 22 18:56:10.770640 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:10.770612 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" event={"ID":"70614419-09b5-4fea-9d3b-aa9bccf3c6c5","Type":"ContainerStarted","Data":"f2c05b80f8a2d7267d6c4c3e1105184ea8a952119d3942311a5198f7207cc287"} Jul 22 18:56:10.792724 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:10.792677 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d96jc" podStartSLOduration=2.685103986 podStartE2EDuration="5.792662851s" totalImagesPullingTime="3.013015637s" totalInitContainerRuntime="94.543228ms" isStatefulPod=true podCreationTimestamp="2026-07-22 18:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:10.791011211 +0000 UTC m=+304.453585551" watchObservedRunningTime="2026-07-22 18:56:10.792662851 +0000 UTC m=+304.455237189" Jul 22 18:56:11.775430 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:11.775339 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" event={"ID":"70614419-09b5-4fea-9d3b-aa9bccf3c6c5","Type":"ContainerStarted","Data":"9a2ee9ad0243323e48b578ae27799b5f8fc2f89e2eaa2e6c50ae96bb4c84a519"} Jul 22 18:56:11.792471 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:11.792423 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" podStartSLOduration=1.500386378 podStartE2EDuration="2.79240862s" totalImagesPullingTime="1.292022242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:11.792075034 +0000 UTC m=+305.454649374" watchObservedRunningTime="2026-07-22 18:56:11.79240862 +0000 UTC m=+305.454982959" Jul 22 18:56:13.558977 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.558931 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" podUID="33072af6-3ef5-40e7-963a-7dc57d1df61c" containerName="registry" containerID="cri-o://3d180acb6f23bd281e57061957ec27187a1b743c0700d36cc09e0030737972d2" gracePeriod=30 Jul 22 18:56:13.785150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.785123 2584 generic.go:356] "Generic (PLEG): container finished" podID="33072af6-3ef5-40e7-963a-7dc57d1df61c" containerID="3d180acb6f23bd281e57061957ec27187a1b743c0700d36cc09e0030737972d2" exitCode=0 Jul 22 18:56:13.785275 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.785205 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" event={"ID":"33072af6-3ef5-40e7-963a-7dc57d1df61c","Type":"ContainerDied","Data":"3d180acb6f23bd281e57061957ec27187a1b743c0700d36cc09e0030737972d2"} Jul 22 18:56:13.799195 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.799172 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:56:13.868645 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.868622 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls\") pod \"33072af6-3ef5-40e7-963a-7dc57d1df61c\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " Jul 22 18:56:13.868769 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.868655 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33072af6-3ef5-40e7-963a-7dc57d1df61c-ca-trust-extracted\") pod \"33072af6-3ef5-40e7-963a-7dc57d1df61c\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " Jul 22 18:56:13.868769 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.868680 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lc98n\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-kube-api-access-lc98n\") pod \"33072af6-3ef5-40e7-963a-7dc57d1df61c\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " Jul 22 18:56:13.868769 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.868699 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-trusted-ca\") pod \"33072af6-3ef5-40e7-963a-7dc57d1df61c\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " Jul 22 18:56:13.868769 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.868741 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-bound-sa-token\") pod \"33072af6-3ef5-40e7-963a-7dc57d1df61c\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " Jul 22 18:56:13.868769 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.868757 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-certificates\") pod \"33072af6-3ef5-40e7-963a-7dc57d1df61c\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " Jul 22 18:56:13.869053 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.868802 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-image-registry-private-configuration\") pod \"33072af6-3ef5-40e7-963a-7dc57d1df61c\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " Jul 22 18:56:13.869053 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.868829 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-installation-pull-secrets\") pod \"33072af6-3ef5-40e7-963a-7dc57d1df61c\" (UID: \"33072af6-3ef5-40e7-963a-7dc57d1df61c\") " Jul 22 18:56:13.869375 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.869311 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "33072af6-3ef5-40e7-963a-7dc57d1df61c" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:56:13.869490 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.869370 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "33072af6-3ef5-40e7-963a-7dc57d1df61c" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:56:13.871480 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.871426 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "33072af6-3ef5-40e7-963a-7dc57d1df61c" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:56:13.871596 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.871530 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-kube-api-access-lc98n" (OuterVolumeSpecName: "kube-api-access-lc98n") pod "33072af6-3ef5-40e7-963a-7dc57d1df61c" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c"). InnerVolumeSpecName "kube-api-access-lc98n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:56:13.871656 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.871601 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "33072af6-3ef5-40e7-963a-7dc57d1df61c" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:56:13.871656 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.871620 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "33072af6-3ef5-40e7-963a-7dc57d1df61c" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:56:13.871766 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.871729 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "33072af6-3ef5-40e7-963a-7dc57d1df61c" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:56:13.877192 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.877167 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33072af6-3ef5-40e7-963a-7dc57d1df61c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "33072af6-3ef5-40e7-963a-7dc57d1df61c" (UID: "33072af6-3ef5-40e7-963a-7dc57d1df61c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:56:13.970129 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.970096 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc98n\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-kube-api-access-lc98n\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:56:13.970129 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.970129 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-trusted-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:56:13.970129 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.970140 2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-bound-sa-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:56:13.970286 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.970147 2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-certificates\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:56:13.970286 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.970156 2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-image-registry-private-configuration\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:56:13.970286 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.970165 2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33072af6-3ef5-40e7-963a-7dc57d1df61c-installation-pull-secrets\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:56:13.970286 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.970172 2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33072af6-3ef5-40e7-963a-7dc57d1df61c-registry-tls\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:56:13.970286 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:13.970180 2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33072af6-3ef5-40e7-963a-7dc57d1df61c-ca-trust-extracted\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:56:14.790439 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:14.790401 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" event={"ID":"33072af6-3ef5-40e7-963a-7dc57d1df61c","Type":"ContainerDied","Data":"dd9e8f46c2f73dc9760e92aa7229173d9665e42ebc5efbb2ae8101b190eef36b"} Jul 22 18:56:14.790439 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:14.790428 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c5889575d-xwd2v" Jul 22 18:56:14.790922 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:14.790485 2584 scope.go:117] "RemoveContainer" containerID="3d180acb6f23bd281e57061957ec27187a1b743c0700d36cc09e0030737972d2" Jul 22 18:56:14.811333 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:14.811308 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c5889575d-xwd2v"] Jul 22 18:56:14.814449 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:14.814428 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c5889575d-xwd2v"] Jul 22 18:56:14.905080 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:14.905058 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33072af6-3ef5-40e7-963a-7dc57d1df61c" path="/var/lib/kubelet/pods/33072af6-3ef5-40e7-963a-7dc57d1df61c/volumes" Jul 22 18:56:17.900230 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:17.900200 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:56:18.900758 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:18.900726 2584 scope.go:117] "RemoveContainer" containerID="f054cad89f71f3c8d17c83653439d44610fb7efe78017b5b6f412508387a7d5b" Jul 22 18:56:19.537712 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.537670 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-tjm7d"] Jul 22 18:56:19.537934 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.537919 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33072af6-3ef5-40e7-963a-7dc57d1df61c" containerName="registry" Jul 22 18:56:19.537971 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.537935 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="33072af6-3ef5-40e7-963a-7dc57d1df61c" containerName="registry" Jul 22 18:56:19.538001 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.537984 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="33072af6-3ef5-40e7-963a-7dc57d1df61c" containerName="registry" Jul 22 18:56:19.540993 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.540974 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-tjm7d" Jul 22 18:56:19.543902 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.543881 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 18:56:19.543990 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.543881 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 18:56:19.543990 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.543965 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-frtlg\"" Jul 22 18:56:19.560118 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.558428 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-tjm7d"] Jul 22 18:56:19.614973 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.614944 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpsfr\" (UniqueName: \"kubernetes.io/projected/f564341b-cc25-48df-92cd-145b2b762c76-kube-api-access-bpsfr\") pod \"downloads-5bc67dcd46-tjm7d\" (UID: \"f564341b-cc25-48df-92cd-145b2b762c76\") " pod="openshift-console/downloads-5bc67dcd46-tjm7d" Jul 22 18:56:19.716337 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.716311 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpsfr\" (UniqueName: \"kubernetes.io/projected/f564341b-cc25-48df-92cd-145b2b762c76-kube-api-access-bpsfr\") pod \"downloads-5bc67dcd46-tjm7d\" (UID: \"f564341b-cc25-48df-92cd-145b2b762c76\") " pod="openshift-console/downloads-5bc67dcd46-tjm7d" Jul 22 18:56:19.724585 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.724558 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpsfr\" (UniqueName: \"kubernetes.io/projected/f564341b-cc25-48df-92cd-145b2b762c76-kube-api-access-bpsfr\") pod \"downloads-5bc67dcd46-tjm7d\" (UID: \"f564341b-cc25-48df-92cd-145b2b762c76\") " pod="openshift-console/downloads-5bc67dcd46-tjm7d" Jul 22 18:56:19.810666 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.810597 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 18:56:19.810794 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.810693 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" event={"ID":"7547bde7-d558-4a05-bdad-e06e98fdcb2a","Type":"ContainerStarted","Data":"6e080a43b8c5487be919b59bfd7879ea6454f152a44cdd8c2aa424862076b54c"} Jul 22 18:56:19.810984 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.810963 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:56:19.815665 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.815641 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" Jul 22 18:56:19.828572 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.828501 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-lqrlz" podStartSLOduration=58.05112695 podStartE2EDuration="59.828487628s" totalImagesPullingTime="1.777360678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:19.828045557 +0000 UTC m=+313.490619898" watchObservedRunningTime="2026-07-22 18:56:19.828487628 +0000 UTC m=+313.491061966" Jul 22 18:56:19.850986 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.850957 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-tjm7d" Jul 22 18:56:19.980737 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:19.980449 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-tjm7d"] Jul 22 18:56:19.983033 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:56:19.983004 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf564341b_cc25_48df_92cd_145b2b762c76.slice/crio-241d4d44f5a7c4b51cade7f166eea4acc6b775edc2b368969ce6ea4663a125ae WatchSource:0}: Error finding container 241d4d44f5a7c4b51cade7f166eea4acc6b775edc2b368969ce6ea4663a125ae: Status 404 returned error can't find the container with id 241d4d44f5a7c4b51cade7f166eea4acc6b775edc2b368969ce6ea4663a125ae Jul 22 18:56:20.816317 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:20.816275 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-tjm7d" event={"ID":"f564341b-cc25-48df-92cd-145b2b762c76","Type":"ContainerStarted","Data":"241d4d44f5a7c4b51cade7f166eea4acc6b775edc2b368969ce6ea4663a125ae"} Jul 22 18:56:29.924206 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:29.924175 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:29.924743 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:29.924259 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:30.460238 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.460206 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-89dbdf445-w7j2d"] Jul 22 18:56:30.462799 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.462772 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.466343 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.466321 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 18:56:30.466467 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.466388 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 18:56:30.466467 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.466384 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 18:56:30.466626 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.466480 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 18:56:30.466626 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.466485 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 18:56:30.466626 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.466493 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-85tjs\"" Jul 22 18:56:30.473010 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.472958 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-89dbdf445-w7j2d"] Jul 22 18:56:30.613427 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.613386 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-oauth-config\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.613631 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.613457 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zw8v\" (UniqueName: \"kubernetes.io/projected/6441577d-c52d-4ae2-8357-f55bf52a6d95-kube-api-access-6zw8v\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.613631 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.613536 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-serving-cert\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.613631 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.613573 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-config\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.613631 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.613599 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-oauth-serving-cert\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.613631 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.613626 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-service-ca\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.714634 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.714551 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-oauth-config\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.714634 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.714607 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zw8v\" (UniqueName: \"kubernetes.io/projected/6441577d-c52d-4ae2-8357-f55bf52a6d95-kube-api-access-6zw8v\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.714855 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.714672 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-serving-cert\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.714855 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.714710 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-config\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.714855 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.714734 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-oauth-serving-cert\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.715027 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.715000 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-service-ca\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.715582 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.715507 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-config\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.715582 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.715545 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-oauth-serving-cert\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.715759 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.715688 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-service-ca\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.717873 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.717853 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-oauth-config\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.718547 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.718502 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-serving-cert\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.723578 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.723551 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zw8v\" (UniqueName: \"kubernetes.io/projected/6441577d-c52d-4ae2-8357-f55bf52a6d95-kube-api-access-6zw8v\") pod \"console-89dbdf445-w7j2d\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:30.775445 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:30.775416 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:36.858583 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:36.858557 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-89dbdf445-w7j2d"] Jul 22 18:56:36.861780 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:56:36.861751 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6441577d_c52d_4ae2_8357_f55bf52a6d95.slice/crio-e0743298858e7f51e76a987d12cc29c3d8362eac2a50dd07c3402cc15ef8e4a9 WatchSource:0}: Error finding container e0743298858e7f51e76a987d12cc29c3d8362eac2a50dd07c3402cc15ef8e4a9: Status 404 returned error can't find the container with id e0743298858e7f51e76a987d12cc29c3d8362eac2a50dd07c3402cc15ef8e4a9 Jul 22 18:56:36.879681 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:36.879650 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-89dbdf445-w7j2d" event={"ID":"6441577d-c52d-4ae2-8357-f55bf52a6d95","Type":"ContainerStarted","Data":"e0743298858e7f51e76a987d12cc29c3d8362eac2a50dd07c3402cc15ef8e4a9"} Jul 22 18:56:37.893648 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:37.893410 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-tjm7d" event={"ID":"f564341b-cc25-48df-92cd-145b2b762c76","Type":"ContainerStarted","Data":"c3be845086920c9aee3a4fd9b3bce968079223cde4fc2474664c79a5d259950b"} Jul 22 18:56:37.894128 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:37.893919 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-tjm7d" Jul 22 18:56:37.906375 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:37.906345 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-tjm7d" Jul 22 18:56:37.913371 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:37.913276 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-tjm7d" podStartSLOduration=2.068196506 podStartE2EDuration="18.913259069s" totalImagesPullingTime="16.845062563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:37.910532267 +0000 UTC m=+331.573106609" watchObservedRunningTime="2026-07-22 18:56:37.913259069 +0000 UTC m=+331.575833409" Jul 22 18:56:39.849721 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.849633 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-645bd99d-5jwl5"] Jul 22 18:56:39.900561 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.900504 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645bd99d-5jwl5"] Jul 22 18:56:39.900743 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.900684 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:39.909763 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.909732 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 18:56:39.996110 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.996069 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-serving-cert\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:39.996110 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.996121 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-service-ca\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:39.996615 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.996148 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvhf2\" (UniqueName: \"kubernetes.io/projected/0e1a649d-74ee-44f4-8411-473b21157b73-kube-api-access-lvhf2\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:39.996615 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.996185 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-oauth-serving-cert\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:39.996615 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.996257 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-trusted-ca-bundle\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:39.996615 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.996316 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-console-config\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:39.996615 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:39.996364 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-oauth-config\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.097569 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.097511 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-console-config\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.097754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.097600 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-oauth-config\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.097754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.097644 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-serving-cert\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.097754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.097664 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-service-ca\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.097754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.097694 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvhf2\" (UniqueName: \"kubernetes.io/projected/0e1a649d-74ee-44f4-8411-473b21157b73-kube-api-access-lvhf2\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.097754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.097730 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-oauth-serving-cert\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.098015 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.097796 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-trusted-ca-bundle\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.098447 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.098395 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-console-config\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.098868 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.098806 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-service-ca\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.098868 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.098835 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-oauth-serving-cert\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.099033 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.098910 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-trusted-ca-bundle\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.100739 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.100682 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-serving-cert\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.100856 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.100784 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-oauth-config\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.106503 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.106476 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvhf2\" (UniqueName: \"kubernetes.io/projected/0e1a649d-74ee-44f4-8411-473b21157b73-kube-api-access-lvhf2\") pod \"console-645bd99d-5jwl5\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.217085 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.217048 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:40.392792 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.392763 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645bd99d-5jwl5"] Jul 22 18:56:40.399946 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:56:40.399913 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1a649d_74ee_44f4_8411_473b21157b73.slice/crio-bc1e3ae9caf85e172a77a186475572f102027e513ec1a516a47109fed87f0a20 WatchSource:0}: Error finding container bc1e3ae9caf85e172a77a186475572f102027e513ec1a516a47109fed87f0a20: Status 404 returned error can't find the container with id bc1e3ae9caf85e172a77a186475572f102027e513ec1a516a47109fed87f0a20 Jul 22 18:56:40.909194 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.909154 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-89dbdf445-w7j2d" event={"ID":"6441577d-c52d-4ae2-8357-f55bf52a6d95","Type":"ContainerStarted","Data":"b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6"} Jul 22 18:56:40.911075 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.911044 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645bd99d-5jwl5" event={"ID":"0e1a649d-74ee-44f4-8411-473b21157b73","Type":"ContainerStarted","Data":"bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7"} Jul 22 18:56:40.911204 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.911087 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645bd99d-5jwl5" event={"ID":"0e1a649d-74ee-44f4-8411-473b21157b73","Type":"ContainerStarted","Data":"bc1e3ae9caf85e172a77a186475572f102027e513ec1a516a47109fed87f0a20"} Jul 22 18:56:40.927585 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.927536 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-89dbdf445-w7j2d" podStartSLOduration=7.368716494 podStartE2EDuration="10.927498808s" totalImagesPullingTime="3.558782314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:40.926902859 +0000 UTC m=+334.589477200" watchObservedRunningTime="2026-07-22 18:56:40.927498808 +0000 UTC m=+334.590073147" Jul 22 18:56:40.949676 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:40.949634 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-645bd99d-5jwl5" podStartSLOduration=1.949619941 podStartE2EDuration="1.949619941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:56:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:56:40.948471182 +0000 UTC m=+334.611045522" watchObservedRunningTime="2026-07-22 18:56:40.949619941 +0000 UTC m=+334.612194279" Jul 22 18:56:49.929315 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:49.929286 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:49.933132 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:49.933111 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8bd746b97-mxgnr" Jul 22 18:56:50.217225 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:50.217147 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:50.217225 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:50.217213 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:50.221763 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:50.221742 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:50.776223 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:50.776186 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:50.776386 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:50.776249 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:50.780681 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:50.780658 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:50.950573 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:50.950544 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:56:50.950975 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:50.950959 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:56:51.013234 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:56:51.013202 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-89dbdf445-w7j2d"] Jul 22 18:57:17.973625 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:17.973589 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-89dbdf445-w7j2d" podUID="6441577d-c52d-4ae2-8357-f55bf52a6d95" containerName="console" containerID="cri-o://b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6" gracePeriod=15 Jul 22 18:57:18.245920 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.245854 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:57:18.408318 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.408271 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zw8v\" (UniqueName: \"kubernetes.io/projected/6441577d-c52d-4ae2-8357-f55bf52a6d95-kube-api-access-6zw8v\") pod \"6441577d-c52d-4ae2-8357-f55bf52a6d95\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " Jul 22 18:57:18.408500 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.408339 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-serving-cert\") pod \"6441577d-c52d-4ae2-8357-f55bf52a6d95\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " Jul 22 18:57:18.408500 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.408388 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-oauth-serving-cert\") pod \"6441577d-c52d-4ae2-8357-f55bf52a6d95\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " Jul 22 18:57:18.408500 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.408422 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-oauth-config\") pod \"6441577d-c52d-4ae2-8357-f55bf52a6d95\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " Jul 22 18:57:18.408500 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.408442 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-service-ca\") pod \"6441577d-c52d-4ae2-8357-f55bf52a6d95\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " Jul 22 18:57:18.408500 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.408470 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-config\") pod \"6441577d-c52d-4ae2-8357-f55bf52a6d95\" (UID: \"6441577d-c52d-4ae2-8357-f55bf52a6d95\") " Jul 22 18:57:18.408888 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.408858 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6441577d-c52d-4ae2-8357-f55bf52a6d95" (UID: "6441577d-c52d-4ae2-8357-f55bf52a6d95"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:57:18.408959 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.408876 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-service-ca" (OuterVolumeSpecName: "service-ca") pod "6441577d-c52d-4ae2-8357-f55bf52a6d95" (UID: "6441577d-c52d-4ae2-8357-f55bf52a6d95"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:57:18.408959 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.408922 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-config" (OuterVolumeSpecName: "console-config") pod "6441577d-c52d-4ae2-8357-f55bf52a6d95" (UID: "6441577d-c52d-4ae2-8357-f55bf52a6d95"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:57:18.410817 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.410791 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6441577d-c52d-4ae2-8357-f55bf52a6d95" (UID: "6441577d-c52d-4ae2-8357-f55bf52a6d95"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:57:18.410936 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.410909 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6441577d-c52d-4ae2-8357-f55bf52a6d95" (UID: "6441577d-c52d-4ae2-8357-f55bf52a6d95"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:57:18.410991 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.410974 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6441577d-c52d-4ae2-8357-f55bf52a6d95-kube-api-access-6zw8v" (OuterVolumeSpecName: "kube-api-access-6zw8v") pod "6441577d-c52d-4ae2-8357-f55bf52a6d95" (UID: "6441577d-c52d-4ae2-8357-f55bf52a6d95"). InnerVolumeSpecName "kube-api-access-6zw8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:57:18.509729 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.509652 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-config\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:57:18.509729 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.509677 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zw8v\" (UniqueName: \"kubernetes.io/projected/6441577d-c52d-4ae2-8357-f55bf52a6d95-kube-api-access-6zw8v\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:57:18.509729 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.509687 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-serving-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:57:18.509729 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.509696 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-oauth-serving-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:57:18.509729 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.509713 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6441577d-c52d-4ae2-8357-f55bf52a6d95-console-oauth-config\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:57:18.509729 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:18.509722 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6441577d-c52d-4ae2-8357-f55bf52a6d95-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:57:19.046780 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:19.046743 2584 generic.go:356] "Generic (PLEG): container finished" podID="6441577d-c52d-4ae2-8357-f55bf52a6d95" containerID="b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6" exitCode=0 Jul 22 18:57:19.047156 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:19.046797 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-89dbdf445-w7j2d" event={"ID":"6441577d-c52d-4ae2-8357-f55bf52a6d95","Type":"ContainerDied","Data":"b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6"} Jul 22 18:57:19.047156 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:19.046838 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-89dbdf445-w7j2d" event={"ID":"6441577d-c52d-4ae2-8357-f55bf52a6d95","Type":"ContainerDied","Data":"e0743298858e7f51e76a987d12cc29c3d8362eac2a50dd07c3402cc15ef8e4a9"} Jul 22 18:57:19.047156 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:19.046858 2584 scope.go:117] "RemoveContainer" containerID="b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6" Jul 22 18:57:19.047156 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:19.046800 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-89dbdf445-w7j2d" Jul 22 18:57:19.054714 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:19.054696 2584 scope.go:117] "RemoveContainer" containerID="b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6" Jul 22 18:57:19.054977 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:57:19.054953 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6\": container with ID starting with b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6 not found: ID does not exist" containerID="b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6" Jul 22 18:57:19.055035 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:19.054986 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6"} err="failed to get container status \"b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6\": rpc error: code = NotFound desc = could not find container \"b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6\": container with ID starting with b0c9ab54795dc490caeb888383cfae9be08be61e80c6b5646615547adb9dc3f6 not found: ID does not exist" Jul 22 18:57:19.062198 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:19.062178 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-89dbdf445-w7j2d"] Jul 22 18:57:19.065682 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:19.065661 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-89dbdf445-w7j2d"] Jul 22 18:57:20.725695 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:20.725662 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:57:20.728084 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:20.728064 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6610a82-7a15-4b2f-9d80-8d2eae96e69e-metrics-certs\") pod \"network-metrics-daemon-l98h2\" (UID: \"d6610a82-7a15-4b2f-9d80-8d2eae96e69e\") " pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:57:20.903420 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:20.903396 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pz9g5\"" Jul 22 18:57:20.905082 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:20.905060 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6441577d-c52d-4ae2-8357-f55bf52a6d95" path="/var/lib/kubelet/pods/6441577d-c52d-4ae2-8357-f55bf52a6d95/volumes" Jul 22 18:57:20.910871 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:20.910855 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l98h2" Jul 22 18:57:21.082424 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:21.082392 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l98h2"] Jul 22 18:57:21.086076 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:57:21.086038 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6610a82_7a15_4b2f_9d80_8d2eae96e69e.slice/crio-697adf715f6e4ec06a19198f13ccd8ef8e971a5b09c635a1dedc8a9df1aa16d2 WatchSource:0}: Error finding container 697adf715f6e4ec06a19198f13ccd8ef8e971a5b09c635a1dedc8a9df1aa16d2: Status 404 returned error can't find the container with id 697adf715f6e4ec06a19198f13ccd8ef8e971a5b09c635a1dedc8a9df1aa16d2 Jul 22 18:57:22.061791 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:22.061754 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l98h2" event={"ID":"d6610a82-7a15-4b2f-9d80-8d2eae96e69e","Type":"ContainerStarted","Data":"697adf715f6e4ec06a19198f13ccd8ef8e971a5b09c635a1dedc8a9df1aa16d2"} Jul 22 18:57:23.067034 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:23.066954 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l98h2" event={"ID":"d6610a82-7a15-4b2f-9d80-8d2eae96e69e","Type":"ContainerStarted","Data":"363cd331544940f25e7a314c4b28d0fae13a9bc8990e3a2a1eb7679365c128a8"} Jul 22 18:57:23.067034 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:23.067002 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l98h2" event={"ID":"d6610a82-7a15-4b2f-9d80-8d2eae96e69e","Type":"ContainerStarted","Data":"a0132ad1e8c475f112b609d02222499ffd6561efaf55d1e7366a00371d6e4abb"} Jul 22 18:57:23.083063 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:23.083025 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l98h2" podStartSLOduration=374.680866729 podStartE2EDuration="6m16.083011249s" totalImagesPullingTime="1.40214452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:23.08183813 +0000 UTC m=+376.744412469" watchObservedRunningTime="2026-07-22 18:57:23.083011249 +0000 UTC m=+376.745585588" Jul 22 18:57:32.538037 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.538004 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bb856cfbd-5md6r"] Jul 22 18:57:32.538475 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.538258 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6441577d-c52d-4ae2-8357-f55bf52a6d95" containerName="console" Jul 22 18:57:32.538475 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.538269 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6441577d-c52d-4ae2-8357-f55bf52a6d95" containerName="console" Jul 22 18:57:32.538475 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.538312 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6441577d-c52d-4ae2-8357-f55bf52a6d95" containerName="console" Jul 22 18:57:32.540150 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.540127 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.551027 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.551003 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bb856cfbd-5md6r"] Jul 22 18:57:32.708434 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.708401 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-serving-cert\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.708434 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.708436 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvlxs\" (UniqueName: \"kubernetes.io/projected/5b186a38-3a97-4923-b7d3-4816c459b3e7-kube-api-access-pvlxs\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.708630 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.708458 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-service-ca\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.708630 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.708589 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-trusted-ca-bundle\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.708695 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.708645 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-config\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.708695 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.708677 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-oauth-config\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.708758 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.708748 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-oauth-serving-cert\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.809818 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.809743 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-serving-cert\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.809818 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.809780 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvlxs\" (UniqueName: \"kubernetes.io/projected/5b186a38-3a97-4923-b7d3-4816c459b3e7-kube-api-access-pvlxs\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.809818 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.809811 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-service-ca\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.810054 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.809850 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-trusted-ca-bundle\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.810054 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.809875 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-config\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.810054 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.809906 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-oauth-config\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.810054 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.810039 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-oauth-serving-cert\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.810740 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.810688 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-trusted-ca-bundle\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.810740 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.810708 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-service-ca\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.810740 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.810708 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-config\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.810958 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.810945 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-oauth-serving-cert\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.812431 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.812403 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-serving-cert\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.812562 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.812481 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-oauth-config\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.818240 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.818217 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvlxs\" (UniqueName: \"kubernetes.io/projected/5b186a38-3a97-4923-b7d3-4816c459b3e7-kube-api-access-pvlxs\") pod \"console-bb856cfbd-5md6r\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.850067 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.850045 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:32.978482 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:32.978459 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bb856cfbd-5md6r"] Jul 22 18:57:32.980368 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:57:32.980335 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b186a38_3a97_4923_b7d3_4816c459b3e7.slice/crio-db5f56db5972f3008f8e58c837f0e04fb6debf322a820e4c8d448fa0f2b4f90d WatchSource:0}: Error finding container db5f56db5972f3008f8e58c837f0e04fb6debf322a820e4c8d448fa0f2b4f90d: Status 404 returned error can't find the container with id db5f56db5972f3008f8e58c837f0e04fb6debf322a820e4c8d448fa0f2b4f90d Jul 22 18:57:33.102912 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:33.102879 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb856cfbd-5md6r" event={"ID":"5b186a38-3a97-4923-b7d3-4816c459b3e7","Type":"ContainerStarted","Data":"67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342"} Jul 22 18:57:33.103067 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:33.102936 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb856cfbd-5md6r" event={"ID":"5b186a38-3a97-4923-b7d3-4816c459b3e7","Type":"ContainerStarted","Data":"db5f56db5972f3008f8e58c837f0e04fb6debf322a820e4c8d448fa0f2b4f90d"} Jul 22 18:57:33.120905 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:33.120859 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bb856cfbd-5md6r" podStartSLOduration=1.120842512 podStartE2EDuration="1.120842512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:57:33.11974313 +0000 UTC m=+386.782317468" watchObservedRunningTime="2026-07-22 18:57:33.120842512 +0000 UTC m=+386.783416850" Jul 22 18:57:42.850802 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:42.850766 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:42.851270 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:42.850843 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:42.855388 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:42.855368 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:43.143859 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:43.143784 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:57:43.186753 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:57:43.186725 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-645bd99d-5jwl5"] Jul 22 18:58:08.212743 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.212693 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-645bd99d-5jwl5" podUID="0e1a649d-74ee-44f4-8411-473b21157b73" containerName="console" containerID="cri-o://bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7" gracePeriod=15 Jul 22 18:58:08.448956 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.448928 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:58:08.567707 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.567632 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-oauth-config\") pod \"0e1a649d-74ee-44f4-8411-473b21157b73\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " Jul 22 18:58:08.567707 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.567669 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-serving-cert\") pod \"0e1a649d-74ee-44f4-8411-473b21157b73\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " Jul 22 18:58:08.567707 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.567700 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-console-config\") pod \"0e1a649d-74ee-44f4-8411-473b21157b73\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " Jul 22 18:58:08.567978 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.567848 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-service-ca\") pod \"0e1a649d-74ee-44f4-8411-473b21157b73\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " Jul 22 18:58:08.567978 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.567909 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvhf2\" (UniqueName: \"kubernetes.io/projected/0e1a649d-74ee-44f4-8411-473b21157b73-kube-api-access-lvhf2\") pod \"0e1a649d-74ee-44f4-8411-473b21157b73\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " Jul 22 18:58:08.567978 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.567950 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-trusted-ca-bundle\") pod \"0e1a649d-74ee-44f4-8411-473b21157b73\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " Jul 22 18:58:08.568120 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.567986 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-oauth-serving-cert\") pod \"0e1a649d-74ee-44f4-8411-473b21157b73\" (UID: \"0e1a649d-74ee-44f4-8411-473b21157b73\") " Jul 22 18:58:08.568263 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.568236 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e1a649d-74ee-44f4-8411-473b21157b73" (UID: "0e1a649d-74ee-44f4-8411-473b21157b73"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:58:08.568350 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.568243 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-console-config" (OuterVolumeSpecName: "console-config") pod "0e1a649d-74ee-44f4-8411-473b21157b73" (UID: "0e1a649d-74ee-44f4-8411-473b21157b73"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:58:08.568350 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.568319 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0e1a649d-74ee-44f4-8411-473b21157b73" (UID: "0e1a649d-74ee-44f4-8411-473b21157b73"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:58:08.568454 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.568400 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e1a649d-74ee-44f4-8411-473b21157b73" (UID: "0e1a649d-74ee-44f4-8411-473b21157b73"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:58:08.570007 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.569978 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e1a649d-74ee-44f4-8411-473b21157b73" (UID: "0e1a649d-74ee-44f4-8411-473b21157b73"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:58:08.570139 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.570082 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1a649d-74ee-44f4-8411-473b21157b73-kube-api-access-lvhf2" (OuterVolumeSpecName: "kube-api-access-lvhf2") pod "0e1a649d-74ee-44f4-8411-473b21157b73" (UID: "0e1a649d-74ee-44f4-8411-473b21157b73"). InnerVolumeSpecName "kube-api-access-lvhf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:58:08.570200 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.570144 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e1a649d-74ee-44f4-8411-473b21157b73" (UID: "0e1a649d-74ee-44f4-8411-473b21157b73"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:58:08.669426 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.669399 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-trusted-ca-bundle\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:58:08.669426 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.669422 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-oauth-serving-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:58:08.669426 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.669432 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-oauth-config\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:58:08.669647 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.669440 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e1a649d-74ee-44f4-8411-473b21157b73-console-serving-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:58:08.669647 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.669449 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-console-config\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:58:08.669647 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.669457 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1a649d-74ee-44f4-8411-473b21157b73-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:58:08.669647 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:08.669465 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvhf2\" (UniqueName: \"kubernetes.io/projected/0e1a649d-74ee-44f4-8411-473b21157b73-kube-api-access-lvhf2\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:58:09.236168 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:09.236083 2584 generic.go:356] "Generic (PLEG): container finished" podID="0e1a649d-74ee-44f4-8411-473b21157b73" containerID="bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7" exitCode=0 Jul 22 18:58:09.236168 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:09.236133 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645bd99d-5jwl5" event={"ID":"0e1a649d-74ee-44f4-8411-473b21157b73","Type":"ContainerDied","Data":"bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7"} Jul 22 18:58:09.236646 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:09.236173 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645bd99d-5jwl5" event={"ID":"0e1a649d-74ee-44f4-8411-473b21157b73","Type":"ContainerDied","Data":"bc1e3ae9caf85e172a77a186475572f102027e513ec1a516a47109fed87f0a20"} Jul 22 18:58:09.236646 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:09.236192 2584 scope.go:117] "RemoveContainer" containerID="bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7" Jul 22 18:58:09.236646 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:09.236137 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645bd99d-5jwl5" Jul 22 18:58:09.246588 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:09.246564 2584 scope.go:117] "RemoveContainer" containerID="bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7" Jul 22 18:58:09.246845 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:58:09.246825 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7\": container with ID starting with bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7 not found: ID does not exist" containerID="bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7" Jul 22 18:58:09.246895 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:09.246857 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7"} err="failed to get container status \"bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7\": rpc error: code = NotFound desc = could not find container \"bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7\": container with ID starting with bf2e37a041c826de3b78a11e76bd6c2c0a51f245a868537f4b0cad108a2e53c7 not found: ID does not exist" Jul 22 18:58:09.252926 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:09.252898 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-645bd99d-5jwl5"] Jul 22 18:58:09.258797 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:09.258778 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-645bd99d-5jwl5"] Jul 22 18:58:10.905037 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:10.904994 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e1a649d-74ee-44f4-8411-473b21157b73" path="/var/lib/kubelet/pods/0e1a649d-74ee-44f4-8411-473b21157b73/volumes" Jul 22 18:58:40.079400 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.079371 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c694df7f9-x82bd"] Jul 22 18:58:40.079869 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.079634 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1a649d-74ee-44f4-8411-473b21157b73" containerName="console" Jul 22 18:58:40.079869 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.079645 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1a649d-74ee-44f4-8411-473b21157b73" containerName="console" Jul 22 18:58:40.079869 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.079697 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1a649d-74ee-44f4-8411-473b21157b73" containerName="console" Jul 22 18:58:40.081353 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.081332 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.095851 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.095829 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c694df7f9-x82bd"] Jul 22 18:58:40.181995 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.181967 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-service-ca\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.182136 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.182001 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-config\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.182136 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.182030 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-serving-cert\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.182136 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.182052 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-oauth-config\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.182136 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.182119 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-oauth-serving-cert\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.182327 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.182171 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-trusted-ca-bundle\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.182327 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.182231 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtm9c\" (UniqueName: \"kubernetes.io/projected/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-kube-api-access-wtm9c\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.283377 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.283352 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-oauth-config\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.283486 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.283384 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-oauth-serving-cert\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.283486 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.283407 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-trusted-ca-bundle\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.283486 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.283427 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtm9c\" (UniqueName: \"kubernetes.io/projected/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-kube-api-access-wtm9c\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.283665 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.283561 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-service-ca\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.283665 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.283599 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-config\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.283665 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.283644 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-serving-cert\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.284187 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.284164 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-oauth-serving-cert\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.284399 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.284380 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-service-ca\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.284561 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.284537 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-config\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.284859 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.284835 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-trusted-ca-bundle\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.286009 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.285991 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-oauth-config\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.286245 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.286228 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-serving-cert\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.292922 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.292893 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtm9c\" (UniqueName: \"kubernetes.io/projected/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-kube-api-access-wtm9c\") pod \"console-6c694df7f9-x82bd\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.391155 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.391099 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:40.522419 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:40.522390 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c694df7f9-x82bd"] Jul 22 18:58:40.525397 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:58:40.525370 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb99d8976_f943_4a5f_8afa_6c00a41ee1e4.slice/crio-bd14caa8efa21e0d7412555bae1521e4f610397b3fb5129631dc5c70b9c72344 WatchSource:0}: Error finding container bd14caa8efa21e0d7412555bae1521e4f610397b3fb5129631dc5c70b9c72344: Status 404 returned error can't find the container with id bd14caa8efa21e0d7412555bae1521e4f610397b3fb5129631dc5c70b9c72344 Jul 22 18:58:41.347064 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:41.347028 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c694df7f9-x82bd" event={"ID":"b99d8976-f943-4a5f-8afa-6c00a41ee1e4","Type":"ContainerStarted","Data":"6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8"} Jul 22 18:58:41.347394 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:41.347076 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c694df7f9-x82bd" event={"ID":"b99d8976-f943-4a5f-8afa-6c00a41ee1e4","Type":"ContainerStarted","Data":"bd14caa8efa21e0d7412555bae1521e4f610397b3fb5129631dc5c70b9c72344"} Jul 22 18:58:41.366715 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:41.366675 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c694df7f9-x82bd" podStartSLOduration=1.366661457 podStartE2EDuration="1.366661457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:58:41.364155413 +0000 UTC m=+455.026729751" watchObservedRunningTime="2026-07-22 18:58:41.366661457 +0000 UTC m=+455.029235795" Jul 22 18:58:50.391673 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:50.391626 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:50.392102 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:50.391721 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:50.396269 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:50.396248 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:51.386257 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:51.386229 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 18:58:51.432754 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:58:51.432714 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bb856cfbd-5md6r"] Jul 22 18:59:16.460865 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.460816 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bb856cfbd-5md6r" podUID="5b186a38-3a97-4923-b7d3-4816c459b3e7" containerName="console" containerID="cri-o://67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342" gracePeriod=15 Jul 22 18:59:16.707991 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.707971 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:59:16.740346 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.740291 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-config\") pod \"5b186a38-3a97-4923-b7d3-4816c459b3e7\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " Jul 22 18:59:16.740458 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.740358 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-oauth-config\") pod \"5b186a38-3a97-4923-b7d3-4816c459b3e7\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " Jul 22 18:59:16.740458 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.740389 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-oauth-serving-cert\") pod \"5b186a38-3a97-4923-b7d3-4816c459b3e7\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " Jul 22 18:59:16.740458 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.740416 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-service-ca\") pod \"5b186a38-3a97-4923-b7d3-4816c459b3e7\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " Jul 22 18:59:16.740458 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.740438 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvlxs\" (UniqueName: \"kubernetes.io/projected/5b186a38-3a97-4923-b7d3-4816c459b3e7-kube-api-access-pvlxs\") pod \"5b186a38-3a97-4923-b7d3-4816c459b3e7\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " Jul 22 18:59:16.740705 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.740471 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-serving-cert\") pod \"5b186a38-3a97-4923-b7d3-4816c459b3e7\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " Jul 22 18:59:16.740705 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.740500 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-trusted-ca-bundle\") pod \"5b186a38-3a97-4923-b7d3-4816c459b3e7\" (UID: \"5b186a38-3a97-4923-b7d3-4816c459b3e7\") " Jul 22 18:59:16.740799 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.740737 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-config" (OuterVolumeSpecName: "console-config") pod "5b186a38-3a97-4923-b7d3-4816c459b3e7" (UID: "5b186a38-3a97-4923-b7d3-4816c459b3e7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:59:16.740939 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.740868 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-service-ca" (OuterVolumeSpecName: "service-ca") pod "5b186a38-3a97-4923-b7d3-4816c459b3e7" (UID: "5b186a38-3a97-4923-b7d3-4816c459b3e7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:59:16.740939 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.740924 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5b186a38-3a97-4923-b7d3-4816c459b3e7" (UID: "5b186a38-3a97-4923-b7d3-4816c459b3e7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:59:16.741075 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.741001 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5b186a38-3a97-4923-b7d3-4816c459b3e7" (UID: "5b186a38-3a97-4923-b7d3-4816c459b3e7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:59:16.742646 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.742627 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5b186a38-3a97-4923-b7d3-4816c459b3e7" (UID: "5b186a38-3a97-4923-b7d3-4816c459b3e7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:59:16.743115 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.743091 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b186a38-3a97-4923-b7d3-4816c459b3e7-kube-api-access-pvlxs" (OuterVolumeSpecName: "kube-api-access-pvlxs") pod "5b186a38-3a97-4923-b7d3-4816c459b3e7" (UID: "5b186a38-3a97-4923-b7d3-4816c459b3e7"). InnerVolumeSpecName "kube-api-access-pvlxs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:59:16.743115 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.743094 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5b186a38-3a97-4923-b7d3-4816c459b3e7" (UID: "5b186a38-3a97-4923-b7d3-4816c459b3e7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:59:16.841453 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.841429 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-config\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:59:16.841453 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.841450 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-oauth-config\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:59:16.841611 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.841460 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-oauth-serving-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:59:16.841611 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.841468 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:59:16.841611 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.841476 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvlxs\" (UniqueName: \"kubernetes.io/projected/5b186a38-3a97-4923-b7d3-4816c459b3e7-kube-api-access-pvlxs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:59:16.841611 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.841484 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b186a38-3a97-4923-b7d3-4816c459b3e7-console-serving-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:59:16.841611 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:16.841492 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b186a38-3a97-4923-b7d3-4816c459b3e7-trusted-ca-bundle\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 18:59:17.473506 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:17.473474 2584 generic.go:356] "Generic (PLEG): container finished" podID="5b186a38-3a97-4923-b7d3-4816c459b3e7" containerID="67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342" exitCode=0 Jul 22 18:59:17.473910 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:17.473538 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb856cfbd-5md6r" Jul 22 18:59:17.473910 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:17.473560 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb856cfbd-5md6r" event={"ID":"5b186a38-3a97-4923-b7d3-4816c459b3e7","Type":"ContainerDied","Data":"67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342"} Jul 22 18:59:17.473910 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:17.473612 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb856cfbd-5md6r" event={"ID":"5b186a38-3a97-4923-b7d3-4816c459b3e7","Type":"ContainerDied","Data":"db5f56db5972f3008f8e58c837f0e04fb6debf322a820e4c8d448fa0f2b4f90d"} Jul 22 18:59:17.473910 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:17.473633 2584 scope.go:117] "RemoveContainer" containerID="67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342" Jul 22 18:59:17.481831 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:17.481817 2584 scope.go:117] "RemoveContainer" containerID="67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342" Jul 22 18:59:17.482041 ip-10-0-142-177 kubenswrapper[2584]: E0722 18:59:17.482019 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342\": container with ID starting with 67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342 not found: ID does not exist" containerID="67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342" Jul 22 18:59:17.482105 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:17.482048 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342"} err="failed to get container status \"67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342\": rpc error: code = NotFound desc = could not find container \"67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342\": container with ID starting with 67f6d30e7c509d7ae06e1248522c8fd53df9beb924157d79cd11c9d6e7676342 not found: ID does not exist" Jul 22 18:59:17.489793 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:17.489775 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bb856cfbd-5md6r"] Jul 22 18:59:17.494108 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:17.494090 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bb856cfbd-5md6r"] Jul 22 18:59:18.905318 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:18.905285 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b186a38-3a97-4923-b7d3-4816c459b3e7" path="/var/lib/kubelet/pods/5b186a38-3a97-4923-b7d3-4816c459b3e7/volumes" Jul 22 18:59:44.162324 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.162293 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v"] Jul 22 18:59:44.162907 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.162568 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b186a38-3a97-4923-b7d3-4816c459b3e7" containerName="console" Jul 22 18:59:44.162907 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.162580 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b186a38-3a97-4923-b7d3-4816c459b3e7" containerName="console" Jul 22 18:59:44.162907 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.162638 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b186a38-3a97-4923-b7d3-4816c459b3e7" containerName="console" Jul 22 18:59:44.165390 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.165372 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" Jul 22 18:59:44.167848 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.167824 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 18:59:44.169285 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.169265 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 18:59:44.169389 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.169343 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 22 18:59:44.169389 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.169344 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 18:59:44.174970 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.174949 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v"] Jul 22 18:59:44.228267 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.228236 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e45ec1be-881d-4832-8a7a-938ea3d7b7d5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v\" (UID: \"e45ec1be-881d-4832-8a7a-938ea3d7b7d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" Jul 22 18:59:44.228387 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.228311 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dshrn\" (UniqueName: \"kubernetes.io/projected/e45ec1be-881d-4832-8a7a-938ea3d7b7d5-kube-api-access-dshrn\") pod \"managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v\" (UID: \"e45ec1be-881d-4832-8a7a-938ea3d7b7d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" Jul 22 18:59:44.272773 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.272746 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq"] Jul 22 18:59:44.275746 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.275731 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.278116 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.278100 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 22 18:59:44.283097 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.283076 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq"] Jul 22 18:59:44.329264 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.329233 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4f393f29-ce03-431a-b97b-74216426850e-klusterlet-config\") pod \"klusterlet-addon-workmgr-78fc7cb4b8-rkzsq\" (UID: \"4f393f29-ce03-431a-b97b-74216426850e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.329365 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.329277 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f393f29-ce03-431a-b97b-74216426850e-tmp\") pod \"klusterlet-addon-workmgr-78fc7cb4b8-rkzsq\" (UID: \"4f393f29-ce03-431a-b97b-74216426850e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.329365 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.329302 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dshrn\" (UniqueName: \"kubernetes.io/projected/e45ec1be-881d-4832-8a7a-938ea3d7b7d5-kube-api-access-dshrn\") pod \"managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v\" (UID: \"e45ec1be-881d-4832-8a7a-938ea3d7b7d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" Jul 22 18:59:44.329365 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.329333 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bpkt\" (UniqueName: \"kubernetes.io/projected/4f393f29-ce03-431a-b97b-74216426850e-kube-api-access-9bpkt\") pod \"klusterlet-addon-workmgr-78fc7cb4b8-rkzsq\" (UID: \"4f393f29-ce03-431a-b97b-74216426850e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.329487 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.329367 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e45ec1be-881d-4832-8a7a-938ea3d7b7d5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v\" (UID: \"e45ec1be-881d-4832-8a7a-938ea3d7b7d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" Jul 22 18:59:44.331874 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.331858 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e45ec1be-881d-4832-8a7a-938ea3d7b7d5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v\" (UID: \"e45ec1be-881d-4832-8a7a-938ea3d7b7d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" Jul 22 18:59:44.337018 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.336999 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dshrn\" (UniqueName: \"kubernetes.io/projected/e45ec1be-881d-4832-8a7a-938ea3d7b7d5-kube-api-access-dshrn\") pod \"managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v\" (UID: \"e45ec1be-881d-4832-8a7a-938ea3d7b7d5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" Jul 22 18:59:44.429831 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.429764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f393f29-ce03-431a-b97b-74216426850e-tmp\") pod \"klusterlet-addon-workmgr-78fc7cb4b8-rkzsq\" (UID: \"4f393f29-ce03-431a-b97b-74216426850e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.429831 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.429795 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bpkt\" (UniqueName: \"kubernetes.io/projected/4f393f29-ce03-431a-b97b-74216426850e-kube-api-access-9bpkt\") pod \"klusterlet-addon-workmgr-78fc7cb4b8-rkzsq\" (UID: \"4f393f29-ce03-431a-b97b-74216426850e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.429970 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.429940 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4f393f29-ce03-431a-b97b-74216426850e-klusterlet-config\") pod \"klusterlet-addon-workmgr-78fc7cb4b8-rkzsq\" (UID: \"4f393f29-ce03-431a-b97b-74216426850e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.430213 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.430191 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f393f29-ce03-431a-b97b-74216426850e-tmp\") pod \"klusterlet-addon-workmgr-78fc7cb4b8-rkzsq\" (UID: \"4f393f29-ce03-431a-b97b-74216426850e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.432560 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.432507 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4f393f29-ce03-431a-b97b-74216426850e-klusterlet-config\") pod \"klusterlet-addon-workmgr-78fc7cb4b8-rkzsq\" (UID: \"4f393f29-ce03-431a-b97b-74216426850e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.439253 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.439231 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bpkt\" (UniqueName: \"kubernetes.io/projected/4f393f29-ce03-431a-b97b-74216426850e-kube-api-access-9bpkt\") pod \"klusterlet-addon-workmgr-78fc7cb4b8-rkzsq\" (UID: \"4f393f29-ce03-431a-b97b-74216426850e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.492625 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.492602 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" Jul 22 18:59:44.585907 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.585882 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:44.622509 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.622478 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v"] Jul 22 18:59:44.625306 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:59:44.625278 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode45ec1be_881d_4832_8a7a_938ea3d7b7d5.slice/crio-ccef52592fb57afeaa206892cb2024f2470f1bdfe0c4d9d46552c61db3caa05d WatchSource:0}: Error finding container ccef52592fb57afeaa206892cb2024f2470f1bdfe0c4d9d46552c61db3caa05d: Status 404 returned error can't find the container with id ccef52592fb57afeaa206892cb2024f2470f1bdfe0c4d9d46552c61db3caa05d Jul 22 18:59:44.716456 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:44.716407 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq"] Jul 22 18:59:44.718800 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:59:44.718776 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f393f29_ce03_431a_b97b_74216426850e.slice/crio-7ff16e170dd06f49551f8ba2999524af3ed0872a82167faafc99356a5f04bcdd WatchSource:0}: Error finding container 7ff16e170dd06f49551f8ba2999524af3ed0872a82167faafc99356a5f04bcdd: Status 404 returned error can't find the container with id 7ff16e170dd06f49551f8ba2999524af3ed0872a82167faafc99356a5f04bcdd Jul 22 18:59:45.572585 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:45.572547 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" event={"ID":"e45ec1be-881d-4832-8a7a-938ea3d7b7d5","Type":"ContainerStarted","Data":"ccef52592fb57afeaa206892cb2024f2470f1bdfe0c4d9d46552c61db3caa05d"} Jul 22 18:59:45.573985 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:45.573952 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" event={"ID":"4f393f29-ce03-431a-b97b-74216426850e","Type":"ContainerStarted","Data":"7ff16e170dd06f49551f8ba2999524af3ed0872a82167faafc99356a5f04bcdd"} Jul 22 18:59:48.586713 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:48.586676 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" event={"ID":"4f393f29-ce03-431a-b97b-74216426850e","Type":"ContainerStarted","Data":"1368fc974c36139c824cf6b0ca99ad768d42212cf0eba96ff7eca8954d03df6c"} Jul 22 18:59:48.587130 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:48.586880 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:48.588413 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:48.588389 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" Jul 22 18:59:48.602686 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:48.602632 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78fc7cb4b8-rkzsq" podStartSLOduration=0.8989122 podStartE2EDuration="4.602615536s" totalImagesPullingTime="3.703703336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:59:48.602217876 +0000 UTC m=+522.264792211" watchObservedRunningTime="2026-07-22 18:59:48.602615536 +0000 UTC m=+522.265189879" Jul 22 18:59:55.735224 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.735189 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg"] Jul 22 18:59:55.738836 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.738816 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:55.741477 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.741457 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 18:59:55.741598 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.741486 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hk26q\"" Jul 22 18:59:55.742535 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.742499 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 18:59:55.747345 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.747321 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg"] Jul 22 18:59:55.813714 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.813685 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:55.813852 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.813737 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jkm5\" (UniqueName: \"kubernetes.io/projected/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-kube-api-access-4jkm5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:55.813852 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.813773 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:55.914386 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.914347 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jkm5\" (UniqueName: \"kubernetes.io/projected/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-kube-api-access-4jkm5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:55.914386 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.914392 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:55.914560 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.914430 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:55.914756 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.914741 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:55.914802 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.914785 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:55.923060 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:55.923038 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jkm5\" (UniqueName: \"kubernetes.io/projected/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-kube-api-access-4jkm5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:56.048563 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:56.048476 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 18:59:56.175909 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:56.175834 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg"] Jul 22 18:59:56.179282 ip-10-0-142-177 kubenswrapper[2584]: W0722 18:59:56.179254 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod842f48e6_21cb_42ad_ad6e_80d4bca7e1dd.slice/crio-71d7672d569a2c85c6e15f8eb15e4ea24f89f36a2bb6f6b195310ed7df3d2c94 WatchSource:0}: Error finding container 71d7672d569a2c85c6e15f8eb15e4ea24f89f36a2bb6f6b195310ed7df3d2c94: Status 404 returned error can't find the container with id 71d7672d569a2c85c6e15f8eb15e4ea24f89f36a2bb6f6b195310ed7df3d2c94 Jul 22 18:59:56.618574 ip-10-0-142-177 kubenswrapper[2584]: I0722 18:59:56.618541 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" event={"ID":"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd","Type":"ContainerStarted","Data":"71d7672d569a2c85c6e15f8eb15e4ea24f89f36a2bb6f6b195310ed7df3d2c94"} Jul 22 19:00:00.635441 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:00.635370 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" event={"ID":"e45ec1be-881d-4832-8a7a-938ea3d7b7d5","Type":"ContainerStarted","Data":"d4989db280ad2296e8900d12dda9fce826906bb2d5d6975dcebae280870bf289"} Jul 22 19:00:00.652646 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:00.652595 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6d79d4b-k4p2v" podStartSLOduration=1.057085845 podStartE2EDuration="16.652576581s" totalImagesPullingTime="15.595490736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:00:00.650237061 +0000 UTC m=+534.312811400" watchObservedRunningTime="2026-07-22 19:00:00.652576581 +0000 UTC m=+534.315150921" Jul 22 19:00:04.651244 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:04.651210 2584 generic.go:356] "Generic (PLEG): container finished" podID="842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" containerID="411fd0f9d75571d10d1c8eaaaa6f3a85a2c2ecb12cc6b71519dbdf4ad54da59e" exitCode=0 Jul 22 19:00:04.651679 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:04.651287 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" event={"ID":"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd","Type":"ContainerDied","Data":"411fd0f9d75571d10d1c8eaaaa6f3a85a2c2ecb12cc6b71519dbdf4ad54da59e"} Jul 22 19:00:10.675484 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:10.675445 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" event={"ID":"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd","Type":"ContainerStarted","Data":"98c2d9ec3d540e4bf0a1bd82d125382c5b42e84220106139ca95b784a8d28720"} Jul 22 19:00:11.680535 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:11.680481 2584 generic.go:356] "Generic (PLEG): container finished" podID="842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" containerID="98c2d9ec3d540e4bf0a1bd82d125382c5b42e84220106139ca95b784a8d28720" exitCode=0 Jul 22 19:00:11.680894 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:11.680560 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" event={"ID":"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd","Type":"ContainerDied","Data":"98c2d9ec3d540e4bf0a1bd82d125382c5b42e84220106139ca95b784a8d28720"} Jul 22 19:00:19.712767 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:19.712733 2584 generic.go:356] "Generic (PLEG): container finished" podID="842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" containerID="fe05e70255a153f85ea19365953309417c79d01493775aa88f3fc7175945f4b5" exitCode=0 Jul 22 19:00:19.713237 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:19.712790 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" event={"ID":"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd","Type":"ContainerDied","Data":"fe05e70255a153f85ea19365953309417c79d01493775aa88f3fc7175945f4b5"} Jul 22 19:00:20.851686 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:20.851660 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 19:00:21.022776 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.022707 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-util\") pod \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " Jul 22 19:00:21.022914 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.022807 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jkm5\" (UniqueName: \"kubernetes.io/projected/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-kube-api-access-4jkm5\") pod \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " Jul 22 19:00:21.022914 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.022831 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-bundle\") pod \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\" (UID: \"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd\") " Jul 22 19:00:21.023557 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.023505 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-bundle" (OuterVolumeSpecName: "bundle") pod "842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" (UID: "842f48e6-21cb-42ad-ad6e-80d4bca7e1dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:00:21.025132 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.025100 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-kube-api-access-4jkm5" (OuterVolumeSpecName: "kube-api-access-4jkm5") pod "842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" (UID: "842f48e6-21cb-42ad-ad6e-80d4bca7e1dd"). InnerVolumeSpecName "kube-api-access-4jkm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:00:21.028273 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.028252 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-util" (OuterVolumeSpecName: "util") pod "842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" (UID: "842f48e6-21cb-42ad-ad6e-80d4bca7e1dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:00:21.123807 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.123773 2584 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-bundle\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:00:21.123807 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.123802 2584 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-util\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:00:21.123807 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.123811 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jkm5\" (UniqueName: \"kubernetes.io/projected/842f48e6-21cb-42ad-ad6e-80d4bca7e1dd-kube-api-access-4jkm5\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:00:21.722137 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.722101 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" event={"ID":"842f48e6-21cb-42ad-ad6e-80d4bca7e1dd","Type":"ContainerDied","Data":"71d7672d569a2c85c6e15f8eb15e4ea24f89f36a2bb6f6b195310ed7df3d2c94"} Jul 22 19:00:21.722137 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.722125 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vxzdg" Jul 22 19:00:21.722343 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:21.722148 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71d7672d569a2c85c6e15f8eb15e4ea24f89f36a2bb6f6b195310ed7df3d2c94" Jul 22 19:00:28.828446 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.828414 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs"] Jul 22 19:00:28.828822 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.828682 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" containerName="pull" Jul 22 19:00:28.828822 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.828693 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" containerName="pull" Jul 22 19:00:28.828822 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.828707 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" containerName="extract" Jul 22 19:00:28.828822 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.828712 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" containerName="extract" Jul 22 19:00:28.828822 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.828722 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" containerName="util" Jul 22 19:00:28.828822 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.828727 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" containerName="util" Jul 22 19:00:28.828822 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.828775 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="842f48e6-21cb-42ad-ad6e-80d4bca7e1dd" containerName="extract" Jul 22 19:00:28.830492 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.830468 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" Jul 22 19:00:28.833313 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.833294 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 22 19:00:28.833625 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.833608 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 22 19:00:28.833725 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.833674 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-kbpwz\"" Jul 22 19:00:28.853061 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.853038 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs"] Jul 22 19:00:28.983573 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.983543 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/039b9bde-4cad-4880-8da7-cda880cb9a54-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x88rs\" (UID: \"039b9bde-4cad-4880-8da7-cda880cb9a54\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" Jul 22 19:00:28.983711 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:28.983603 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgt85\" (UniqueName: \"kubernetes.io/projected/039b9bde-4cad-4880-8da7-cda880cb9a54-kube-api-access-pgt85\") pod \"cert-manager-operator-controller-manager-55c444bf86-x88rs\" (UID: \"039b9bde-4cad-4880-8da7-cda880cb9a54\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" Jul 22 19:00:29.084599 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:29.084570 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/039b9bde-4cad-4880-8da7-cda880cb9a54-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x88rs\" (UID: \"039b9bde-4cad-4880-8da7-cda880cb9a54\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" Jul 22 19:00:29.084751 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:29.084629 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgt85\" (UniqueName: \"kubernetes.io/projected/039b9bde-4cad-4880-8da7-cda880cb9a54-kube-api-access-pgt85\") pod \"cert-manager-operator-controller-manager-55c444bf86-x88rs\" (UID: \"039b9bde-4cad-4880-8da7-cda880cb9a54\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" Jul 22 19:00:29.084970 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:29.084947 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/039b9bde-4cad-4880-8da7-cda880cb9a54-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-x88rs\" (UID: \"039b9bde-4cad-4880-8da7-cda880cb9a54\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" Jul 22 19:00:29.094023 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:29.094003 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgt85\" (UniqueName: \"kubernetes.io/projected/039b9bde-4cad-4880-8da7-cda880cb9a54-kube-api-access-pgt85\") pod \"cert-manager-operator-controller-manager-55c444bf86-x88rs\" (UID: \"039b9bde-4cad-4880-8da7-cda880cb9a54\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" Jul 22 19:00:29.139109 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:29.139087 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" Jul 22 19:00:29.271228 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:29.271202 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs"] Jul 22 19:00:29.272282 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:00:29.272257 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039b9bde_4cad_4880_8da7_cda880cb9a54.slice/crio-b595a56a369efcd6bd4ea5cbb719e35510ef68208799daae50b55e741321fc52 WatchSource:0}: Error finding container b595a56a369efcd6bd4ea5cbb719e35510ef68208799daae50b55e741321fc52: Status 404 returned error can't find the container with id b595a56a369efcd6bd4ea5cbb719e35510ef68208799daae50b55e741321fc52 Jul 22 19:00:29.752263 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:29.752231 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" event={"ID":"039b9bde-4cad-4880-8da7-cda880cb9a54","Type":"ContainerStarted","Data":"b595a56a369efcd6bd4ea5cbb719e35510ef68208799daae50b55e741321fc52"} Jul 22 19:00:34.776606 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:34.776568 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" event={"ID":"039b9bde-4cad-4880-8da7-cda880cb9a54","Type":"ContainerStarted","Data":"5cc855527eb8b46c1661f7e7edb924b5d6ff709bdf47d0ed77681eeaa47c0325"} Jul 22 19:00:34.801854 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:34.801761 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-x88rs" podStartSLOduration=1.543084949 podStartE2EDuration="6.801743327s" totalImagesPullingTime="5.258658378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:00:34.800066208 +0000 UTC m=+568.462640546" watchObservedRunningTime="2026-07-22 19:00:34.801743327 +0000 UTC m=+568.464317655" Jul 22 19:00:36.252642 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.252607 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qtgqk"] Jul 22 19:00:36.254810 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.254791 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" Jul 22 19:00:36.258698 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.258675 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 19:00:36.260017 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.259997 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-dcwm9\"" Jul 22 19:00:36.261836 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.261592 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 19:00:36.275593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.275569 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qtgqk"] Jul 22 19:00:36.344902 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.344875 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qtgqk\" (UID: \"6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" Jul 22 19:00:36.345026 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.344924 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgjql\" (UniqueName: \"kubernetes.io/projected/6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7-kube-api-access-zgjql\") pod \"cert-manager-webhook-6486ffd997-qtgqk\" (UID: \"6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" Jul 22 19:00:36.445783 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.445745 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qtgqk\" (UID: \"6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" Jul 22 19:00:36.445976 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.445796 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgjql\" (UniqueName: \"kubernetes.io/projected/6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7-kube-api-access-zgjql\") pod \"cert-manager-webhook-6486ffd997-qtgqk\" (UID: \"6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" Jul 22 19:00:36.454109 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.454075 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qtgqk\" (UID: \"6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" Jul 22 19:00:36.454350 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.454329 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgjql\" (UniqueName: \"kubernetes.io/projected/6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7-kube-api-access-zgjql\") pod \"cert-manager-webhook-6486ffd997-qtgqk\" (UID: \"6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" Jul 22 19:00:36.564723 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.564646 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" Jul 22 19:00:36.666205 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.666166 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n"] Jul 22 19:00:36.668851 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.668825 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:36.671367 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.671344 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 19:00:36.671492 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.671346 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 19:00:36.671492 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.671369 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hk26q\"" Jul 22 19:00:36.679933 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.679913 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n"] Jul 22 19:00:36.693710 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.693691 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qtgqk"] Jul 22 19:00:36.696072 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:00:36.696046 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f2c6af3_e80c_4c0a_b8d0_2d40731ab3c7.slice/crio-d0cc4b6be3505a35b3c3ac4ae18d2512d87177cea147f59cb76b9a2adc1e5b0e WatchSource:0}: Error finding container d0cc4b6be3505a35b3c3ac4ae18d2512d87177cea147f59cb76b9a2adc1e5b0e: Status 404 returned error can't find the container with id d0cc4b6be3505a35b3c3ac4ae18d2512d87177cea147f59cb76b9a2adc1e5b0e Jul 22 19:00:36.747637 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.747609 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:36.747775 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.747664 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sj8b\" (UniqueName: \"kubernetes.io/projected/5e489cda-98ab-447e-8789-a50240850bdd-kube-api-access-9sj8b\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:36.747775 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.747693 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:36.785440 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.785407 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" event={"ID":"6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7","Type":"ContainerStarted","Data":"d0cc4b6be3505a35b3c3ac4ae18d2512d87177cea147f59cb76b9a2adc1e5b0e"} Jul 22 19:00:36.848818 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.848781 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sj8b\" (UniqueName: \"kubernetes.io/projected/5e489cda-98ab-447e-8789-a50240850bdd-kube-api-access-9sj8b\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:36.848818 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.848819 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:36.849030 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.848864 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:36.849169 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.849154 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:36.849220 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.849200 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:36.857556 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.857512 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sj8b\" (UniqueName: \"kubernetes.io/projected/5e489cda-98ab-447e-8789-a50240850bdd-kube-api-access-9sj8b\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:36.979512 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:36.979482 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:37.112433 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:37.112406 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n"] Jul 22 19:00:37.114792 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:00:37.114770 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e489cda_98ab_447e_8789_a50240850bdd.slice/crio-320690fc7d7807f20f30f606b1c8b5939fababe7f2e45236c35cfb7a5fc1bb23 WatchSource:0}: Error finding container 320690fc7d7807f20f30f606b1c8b5939fababe7f2e45236c35cfb7a5fc1bb23: Status 404 returned error can't find the container with id 320690fc7d7807f20f30f606b1c8b5939fababe7f2e45236c35cfb7a5fc1bb23 Jul 22 19:00:37.791335 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:37.791300 2584 generic.go:356] "Generic (PLEG): container finished" podID="5e489cda-98ab-447e-8789-a50240850bdd" containerID="fc9b0852b0d351dc289c29f6101bf38a4f81bace94d981e9523f974125beb644" exitCode=0 Jul 22 19:00:37.791829 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:37.791360 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" event={"ID":"5e489cda-98ab-447e-8789-a50240850bdd","Type":"ContainerDied","Data":"fc9b0852b0d351dc289c29f6101bf38a4f81bace94d981e9523f974125beb644"} Jul 22 19:00:37.791829 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:37.791413 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" event={"ID":"5e489cda-98ab-447e-8789-a50240850bdd","Type":"ContainerStarted","Data":"320690fc7d7807f20f30f606b1c8b5939fababe7f2e45236c35cfb7a5fc1bb23"} Jul 22 19:00:41.810876 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:41.810840 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" event={"ID":"6f2c6af3-e80c-4c0a-b8d0-2d40731ab3c7","Type":"ContainerStarted","Data":"43d65aa714ee3562e27b38423d38e6fe14a07ea33de59586b71a9827bdcabbce"} Jul 22 19:00:41.810876 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:41.810891 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" Jul 22 19:00:41.812664 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:41.812641 2584 generic.go:356] "Generic (PLEG): container finished" podID="5e489cda-98ab-447e-8789-a50240850bdd" containerID="885a670a7dbeb46bd67a650aecad6b98f49b060cab2b603ac42efcd546068268" exitCode=0 Jul 22 19:00:41.812788 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:41.812681 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" event={"ID":"5e489cda-98ab-447e-8789-a50240850bdd","Type":"ContainerDied","Data":"885a670a7dbeb46bd67a650aecad6b98f49b060cab2b603ac42efcd546068268"} Jul 22 19:00:41.826851 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:41.826298 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" podStartSLOduration=1.35629939 podStartE2EDuration="5.826278114s" totalImagesPullingTime="4.469978724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:00:41.8261785 +0000 UTC m=+575.488752839" watchObservedRunningTime="2026-07-22 19:00:41.826278114 +0000 UTC m=+575.488852453" Jul 22 19:00:42.819831 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:42.819791 2584 generic.go:356] "Generic (PLEG): container finished" podID="5e489cda-98ab-447e-8789-a50240850bdd" containerID="62a4207c00b908db97fcdd9090cc69aaf55a43db6c4c09436f8af4200791b0f0" exitCode=0 Jul 22 19:00:42.820295 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:42.819879 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" event={"ID":"5e489cda-98ab-447e-8789-a50240850bdd","Type":"ContainerDied","Data":"62a4207c00b908db97fcdd9090cc69aaf55a43db6c4c09436f8af4200791b0f0"} Jul 22 19:00:43.958848 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:43.958825 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:44.002075 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.002048 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-util\") pod \"5e489cda-98ab-447e-8789-a50240850bdd\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " Jul 22 19:00:44.002210 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.002085 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sj8b\" (UniqueName: \"kubernetes.io/projected/5e489cda-98ab-447e-8789-a50240850bdd-kube-api-access-9sj8b\") pod \"5e489cda-98ab-447e-8789-a50240850bdd\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " Jul 22 19:00:44.002210 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.002150 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-bundle\") pod \"5e489cda-98ab-447e-8789-a50240850bdd\" (UID: \"5e489cda-98ab-447e-8789-a50240850bdd\") " Jul 22 19:00:44.002610 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.002586 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-bundle" (OuterVolumeSpecName: "bundle") pod "5e489cda-98ab-447e-8789-a50240850bdd" (UID: "5e489cda-98ab-447e-8789-a50240850bdd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:00:44.004367 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.004341 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e489cda-98ab-447e-8789-a50240850bdd-kube-api-access-9sj8b" (OuterVolumeSpecName: "kube-api-access-9sj8b") pod "5e489cda-98ab-447e-8789-a50240850bdd" (UID: "5e489cda-98ab-447e-8789-a50240850bdd"). InnerVolumeSpecName "kube-api-access-9sj8b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:00:44.006681 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.006659 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-util" (OuterVolumeSpecName: "util") pod "5e489cda-98ab-447e-8789-a50240850bdd" (UID: "5e489cda-98ab-447e-8789-a50240850bdd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:00:44.102860 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.102833 2584 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-bundle\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:00:44.102860 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.102855 2584 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e489cda-98ab-447e-8789-a50240850bdd-util\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:00:44.103000 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.102865 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sj8b\" (UniqueName: \"kubernetes.io/projected/5e489cda-98ab-447e-8789-a50240850bdd-kube-api-access-9sj8b\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:00:44.830397 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.830364 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" Jul 22 19:00:44.830562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.830368 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f95x6n" event={"ID":"5e489cda-98ab-447e-8789-a50240850bdd","Type":"ContainerDied","Data":"320690fc7d7807f20f30f606b1c8b5939fababe7f2e45236c35cfb7a5fc1bb23"} Jul 22 19:00:44.830562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:44.830487 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="320690fc7d7807f20f30f606b1c8b5939fababe7f2e45236c35cfb7a5fc1bb23" Jul 22 19:00:47.821678 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:47.821646 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qtgqk" Jul 22 19:00:50.582152 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.582118 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l"] Jul 22 19:00:50.582562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.582382 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e489cda-98ab-447e-8789-a50240850bdd" containerName="extract" Jul 22 19:00:50.582562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.582394 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e489cda-98ab-447e-8789-a50240850bdd" containerName="extract" Jul 22 19:00:50.582562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.582409 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e489cda-98ab-447e-8789-a50240850bdd" containerName="pull" Jul 22 19:00:50.582562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.582415 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e489cda-98ab-447e-8789-a50240850bdd" containerName="pull" Jul 22 19:00:50.582562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.582426 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e489cda-98ab-447e-8789-a50240850bdd" containerName="util" Jul 22 19:00:50.582562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.582431 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e489cda-98ab-447e-8789-a50240850bdd" containerName="util" Jul 22 19:00:50.582562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.582475 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e489cda-98ab-447e-8789-a50240850bdd" containerName="extract" Jul 22 19:00:50.598865 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.598835 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" Jul 22 19:00:50.599011 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.598839 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l"] Jul 22 19:00:50.601578 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.601559 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 19:00:50.602737 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.602709 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-gn99r\"" Jul 22 19:00:50.602838 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.602741 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 19:00:50.646063 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.646032 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ef95fea-e92b-4dd4-9d1f-0b0218a3494d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pcp6l\" (UID: \"9ef95fea-e92b-4dd4-9d1f-0b0218a3494d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" Jul 22 19:00:50.646187 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.646075 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6dl5\" (UniqueName: \"kubernetes.io/projected/9ef95fea-e92b-4dd4-9d1f-0b0218a3494d-kube-api-access-g6dl5\") pod \"openshift-lws-operator-fd8ccff4c-pcp6l\" (UID: \"9ef95fea-e92b-4dd4-9d1f-0b0218a3494d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" Jul 22 19:00:50.747183 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.747150 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ef95fea-e92b-4dd4-9d1f-0b0218a3494d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pcp6l\" (UID: \"9ef95fea-e92b-4dd4-9d1f-0b0218a3494d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" Jul 22 19:00:50.747308 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.747198 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6dl5\" (UniqueName: \"kubernetes.io/projected/9ef95fea-e92b-4dd4-9d1f-0b0218a3494d-kube-api-access-g6dl5\") pod \"openshift-lws-operator-fd8ccff4c-pcp6l\" (UID: \"9ef95fea-e92b-4dd4-9d1f-0b0218a3494d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" Jul 22 19:00:50.747613 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.747596 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ef95fea-e92b-4dd4-9d1f-0b0218a3494d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pcp6l\" (UID: \"9ef95fea-e92b-4dd4-9d1f-0b0218a3494d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" Jul 22 19:00:50.756307 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.756273 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6dl5\" (UniqueName: \"kubernetes.io/projected/9ef95fea-e92b-4dd4-9d1f-0b0218a3494d-kube-api-access-g6dl5\") pod \"openshift-lws-operator-fd8ccff4c-pcp6l\" (UID: \"9ef95fea-e92b-4dd4-9d1f-0b0218a3494d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" Jul 22 19:00:50.909946 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:50.909880 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" Jul 22 19:00:51.069353 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:51.069329 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l"] Jul 22 19:00:51.071436 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:00:51.071406 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ef95fea_e92b_4dd4_9d1f_0b0218a3494d.slice/crio-a238a2ceee1c8943b6742878fcbf94e52f5a4cf6d0605760f9164a33fe7c8df8 WatchSource:0}: Error finding container a238a2ceee1c8943b6742878fcbf94e52f5a4cf6d0605760f9164a33fe7c8df8: Status 404 returned error can't find the container with id a238a2ceee1c8943b6742878fcbf94e52f5a4cf6d0605760f9164a33fe7c8df8 Jul 22 19:00:51.859655 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:51.859616 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" event={"ID":"9ef95fea-e92b-4dd4-9d1f-0b0218a3494d","Type":"ContainerStarted","Data":"a238a2ceee1c8943b6742878fcbf94e52f5a4cf6d0605760f9164a33fe7c8df8"} Jul 22 19:00:54.875287 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:54.875256 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" event={"ID":"9ef95fea-e92b-4dd4-9d1f-0b0218a3494d","Type":"ContainerStarted","Data":"6420cc9e4618f31f39a97f7e8cc72818f1a7da1747f4db71bad5b768d9eaa842"} Jul 22 19:00:54.892718 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:54.892667 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pcp6l" podStartSLOduration=1.853724727 podStartE2EDuration="4.892651427s" totalImagesPullingTime="3.0389267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:00:54.89109021 +0000 UTC m=+588.553664552" watchObservedRunningTime="2026-07-22 19:00:54.892651427 +0000 UTC m=+588.555225769" Jul 22 19:00:57.006335 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.006303 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9"] Jul 22 19:00:57.008752 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.008735 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.011379 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.011347 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 19:00:57.011379 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.011352 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hk26q\"" Jul 22 19:00:57.011542 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.011350 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 19:00:57.017649 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.017629 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9"] Jul 22 19:00:57.101528 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.101468 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9bt7\" (UniqueName: \"kubernetes.io/projected/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-kube-api-access-z9bt7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.101694 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.101605 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.101694 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.101655 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.203067 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.203026 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.203248 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.203107 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9bt7\" (UniqueName: \"kubernetes.io/projected/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-kube-api-access-z9bt7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.203248 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.203212 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.203765 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.203737 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.204054 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.204030 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.211983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.211958 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9bt7\" (UniqueName: \"kubernetes.io/projected/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-kube-api-access-z9bt7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.318066 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.318000 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:00:57.450113 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.450079 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9"] Jul 22 19:00:57.453090 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:00:57.453057 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ced2e93_074a_4f7b_b5b4_9f2bb00964be.slice/crio-12b67a78b5987da7e993c512d2531de22d50d68aaaeea370573fc6aed7df9ebc WatchSource:0}: Error finding container 12b67a78b5987da7e993c512d2531de22d50d68aaaeea370573fc6aed7df9ebc: Status 404 returned error can't find the container with id 12b67a78b5987da7e993c512d2531de22d50d68aaaeea370573fc6aed7df9ebc Jul 22 19:00:57.889629 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.889594 2584 generic.go:356] "Generic (PLEG): container finished" podID="2ced2e93-074a-4f7b-b5b4-9f2bb00964be" containerID="27f9471b56f35273dff65be4afa2204643b0fa324961921f267988451b75ac78" exitCode=0 Jul 22 19:00:57.889829 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.889679 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" event={"ID":"2ced2e93-074a-4f7b-b5b4-9f2bb00964be","Type":"ContainerDied","Data":"27f9471b56f35273dff65be4afa2204643b0fa324961921f267988451b75ac78"} Jul 22 19:00:57.889829 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:57.889734 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" event={"ID":"2ced2e93-074a-4f7b-b5b4-9f2bb00964be","Type":"ContainerStarted","Data":"12b67a78b5987da7e993c512d2531de22d50d68aaaeea370573fc6aed7df9ebc"} Jul 22 19:00:58.895476 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:58.895446 2584 generic.go:356] "Generic (PLEG): container finished" podID="2ced2e93-074a-4f7b-b5b4-9f2bb00964be" containerID="ce13694c8b5622027869125bc33daad5dc83ba81f8f0610628d1fc7ea7e42a2a" exitCode=0 Jul 22 19:00:58.895935 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:58.895563 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" event={"ID":"2ced2e93-074a-4f7b-b5b4-9f2bb00964be","Type":"ContainerDied","Data":"ce13694c8b5622027869125bc33daad5dc83ba81f8f0610628d1fc7ea7e42a2a"} Jul 22 19:00:59.902359 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:59.902327 2584 generic.go:356] "Generic (PLEG): container finished" podID="2ced2e93-074a-4f7b-b5b4-9f2bb00964be" containerID="6fb647faeaa8ad730ceb3a32e319473b4c827e3ed6a0778f00c09ec812ceb3c4" exitCode=0 Jul 22 19:00:59.902753 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:00:59.902400 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" event={"ID":"2ced2e93-074a-4f7b-b5b4-9f2bb00964be","Type":"ContainerDied","Data":"6fb647faeaa8ad730ceb3a32e319473b4c827e3ed6a0778f00c09ec812ceb3c4"} Jul 22 19:01:01.033955 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.033924 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:01:01.132562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.132503 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-bundle\") pod \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " Jul 22 19:01:01.132735 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.132580 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9bt7\" (UniqueName: \"kubernetes.io/projected/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-kube-api-access-z9bt7\") pod \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " Jul 22 19:01:01.132735 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.132607 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-util\") pod \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\" (UID: \"2ced2e93-074a-4f7b-b5b4-9f2bb00964be\") " Jul 22 19:01:01.133294 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.133270 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-bundle" (OuterVolumeSpecName: "bundle") pod "2ced2e93-074a-4f7b-b5b4-9f2bb00964be" (UID: "2ced2e93-074a-4f7b-b5b4-9f2bb00964be"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:01:01.134831 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.134806 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-kube-api-access-z9bt7" (OuterVolumeSpecName: "kube-api-access-z9bt7") pod "2ced2e93-074a-4f7b-b5b4-9f2bb00964be" (UID: "2ced2e93-074a-4f7b-b5b4-9f2bb00964be"). InnerVolumeSpecName "kube-api-access-z9bt7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:01:01.138771 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.138720 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-util" (OuterVolumeSpecName: "util") pod "2ced2e93-074a-4f7b-b5b4-9f2bb00964be" (UID: "2ced2e93-074a-4f7b-b5b4-9f2bb00964be"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:01:01.233695 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.233611 2584 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-bundle\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:01:01.233695 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.233638 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9bt7\" (UniqueName: \"kubernetes.io/projected/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-kube-api-access-z9bt7\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:01:01.233695 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.233653 2584 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ced2e93-074a-4f7b-b5b4-9f2bb00964be-util\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:01:01.916957 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.916930 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" Jul 22 19:01:01.916957 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.916951 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afstht9" event={"ID":"2ced2e93-074a-4f7b-b5b4-9f2bb00964be","Type":"ContainerDied","Data":"12b67a78b5987da7e993c512d2531de22d50d68aaaeea370573fc6aed7df9ebc"} Jul 22 19:01:01.917163 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:01.917013 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12b67a78b5987da7e993c512d2531de22d50d68aaaeea370573fc6aed7df9ebc" Jul 22 19:01:09.131354 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:09.131331 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:01:09.132929 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:09.131364 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:01:10.967880 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.967845 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8"] Jul 22 19:01:10.968276 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.968191 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ced2e93-074a-4f7b-b5b4-9f2bb00964be" containerName="pull" Jul 22 19:01:10.968276 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.968206 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ced2e93-074a-4f7b-b5b4-9f2bb00964be" containerName="pull" Jul 22 19:01:10.968276 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.968225 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ced2e93-074a-4f7b-b5b4-9f2bb00964be" containerName="extract" Jul 22 19:01:10.968276 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.968231 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ced2e93-074a-4f7b-b5b4-9f2bb00964be" containerName="extract" Jul 22 19:01:10.968276 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.968239 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ced2e93-074a-4f7b-b5b4-9f2bb00964be" containerName="util" Jul 22 19:01:10.968276 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.968245 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ced2e93-074a-4f7b-b5b4-9f2bb00964be" containerName="util" Jul 22 19:01:10.968470 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.968294 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ced2e93-074a-4f7b-b5b4-9f2bb00964be" containerName="extract" Jul 22 19:01:10.970350 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.970335 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:10.973296 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.973263 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-snk6r\"" Jul 22 19:01:10.973296 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.973249 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 19:01:10.973296 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.973287 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 19:01:10.973487 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.973289 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 19:01:10.973487 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.973455 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 19:01:10.994294 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:10.994273 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8"] Jul 22 19:01:11.114342 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.114315 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-webhook-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.114456 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.114351 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbfv9\" (UniqueName: \"kubernetes.io/projected/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-kube-api-access-xbfv9\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.114456 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.114376 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.114456 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.114437 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-tmp\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.114588 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.114472 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-manifests\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.214810 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.214781 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-webhook-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.214924 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.214821 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbfv9\" (UniqueName: \"kubernetes.io/projected/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-kube-api-access-xbfv9\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.214924 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.214848 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.214924 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.214867 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-tmp\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.215025 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.214979 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-manifests\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.215378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.215360 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-manifests\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.217277 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.217255 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-tmp\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.217668 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.217647 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.217714 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.217647 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-webhook-cert\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.226628 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.226581 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbfv9\" (UniqueName: \"kubernetes.io/projected/9c181f18-bdc9-4bb2-a871-37259a0ad2f9-kube-api-access-xbfv9\") pod \"opendatahub-operator-controller-manager-64cd98d4d8-nt5d8\" (UID: \"9c181f18-bdc9-4bb2-a871-37259a0ad2f9\") " pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.280195 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.280164 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:11.415914 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.415887 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8"] Jul 22 19:01:11.417554 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:01:11.417501 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c181f18_bdc9_4bb2_a871_37259a0ad2f9.slice/crio-965a6c4c9dfaa1edd799773c28b4053835b43155d6d49900be4d616ca6facc43 WatchSource:0}: Error finding container 965a6c4c9dfaa1edd799773c28b4053835b43155d6d49900be4d616ca6facc43: Status 404 returned error can't find the container with id 965a6c4c9dfaa1edd799773c28b4053835b43155d6d49900be4d616ca6facc43 Jul 22 19:01:11.419608 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.419588 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 19:01:11.958699 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:11.958662 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" event={"ID":"9c181f18-bdc9-4bb2-a871-37259a0ad2f9","Type":"ContainerStarted","Data":"965a6c4c9dfaa1edd799773c28b4053835b43155d6d49900be4d616ca6facc43"} Jul 22 19:01:14.974792 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:14.974763 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" event={"ID":"9c181f18-bdc9-4bb2-a871-37259a0ad2f9","Type":"ContainerStarted","Data":"e455a30e14e3777ca2647cd377a86233a74d86eefc85b2962b390363be1317c2"} Jul 22 19:01:16.984584 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:16.984548 2584 generic.go:356] "Generic (PLEG): container finished" podID="9c181f18-bdc9-4bb2-a871-37259a0ad2f9" containerID="e455a30e14e3777ca2647cd377a86233a74d86eefc85b2962b390363be1317c2" exitCode=0 Jul 22 19:01:16.984584 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:16.984586 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" event={"ID":"9c181f18-bdc9-4bb2-a871-37259a0ad2f9","Type":"ContainerDied","Data":"e455a30e14e3777ca2647cd377a86233a74d86eefc85b2962b390363be1317c2"} Jul 22 19:01:17.990701 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:17.990667 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" event={"ID":"9c181f18-bdc9-4bb2-a871-37259a0ad2f9","Type":"ContainerStarted","Data":"63786f7a997c84dd5f3c4870b46b618cd3f675a7cdacf6b64edd24114e3af580"} Jul 22 19:01:17.991083 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:17.990879 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:18.013590 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:18.013545 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" podStartSLOduration=3.065242932 podStartE2EDuration="8.013512265s" totalImagesPullingTime="3.909778809s" totalInitContainerRuntime="1.038490524s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:18.012092696 +0000 UTC m=+611.674667038" watchObservedRunningTime="2026-07-22 19:01:18.013512265 +0000 UTC m=+611.676086603" Jul 22 19:01:28.996637 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:28.996569 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64cd98d4d8-nt5d8" Jul 22 19:01:29.807875 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.807845 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk"] Jul 22 19:01:29.810126 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.810104 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:29.812824 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.812805 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 19:01:29.814024 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.814006 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 19:01:29.814149 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.814132 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vgq9z\"" Jul 22 19:01:29.814275 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.814256 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 19:01:29.819025 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.819001 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk"] Jul 22 19:01:29.973188 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.973153 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c38839a-dab6-4cbe-b004-641a60a1138a-cert\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:29.973335 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.973195 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b6v4\" (UniqueName: \"kubernetes.io/projected/8c38839a-dab6-4cbe-b004-641a60a1138a-kube-api-access-7b6v4\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:29.973335 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.973249 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c38839a-dab6-4cbe-b004-641a60a1138a-metrics-cert\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:29.973335 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:29.973276 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8c38839a-dab6-4cbe-b004-641a60a1138a-manager-config\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:30.073696 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:30.073618 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c38839a-dab6-4cbe-b004-641a60a1138a-cert\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:30.073696 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:30.073656 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b6v4\" (UniqueName: \"kubernetes.io/projected/8c38839a-dab6-4cbe-b004-641a60a1138a-kube-api-access-7b6v4\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:30.074160 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:30.073707 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c38839a-dab6-4cbe-b004-641a60a1138a-metrics-cert\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:30.074160 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:30.073754 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8c38839a-dab6-4cbe-b004-641a60a1138a-manager-config\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:30.074391 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:30.074371 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8c38839a-dab6-4cbe-b004-641a60a1138a-manager-config\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:30.076324 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:30.076291 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c38839a-dab6-4cbe-b004-641a60a1138a-metrics-cert\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:30.076433 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:30.076334 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c38839a-dab6-4cbe-b004-641a60a1138a-cert\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:30.081865 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:30.081845 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b6v4\" (UniqueName: \"kubernetes.io/projected/8c38839a-dab6-4cbe-b004-641a60a1138a-kube-api-access-7b6v4\") pod \"lws-controller-manager-5c9874fb98-gt9tk\" (UID: \"8c38839a-dab6-4cbe-b004-641a60a1138a\") " pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:30.120464 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:30.120443 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:30.249552 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:30.249504 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk"] Jul 22 19:01:30.252624 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:01:30.252595 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c38839a_dab6_4cbe_b004_641a60a1138a.slice/crio-039fe7dcc8471eb18319a1b4e7783ed5fdb4a43e87acdef7fa71955eb05b1a88 WatchSource:0}: Error finding container 039fe7dcc8471eb18319a1b4e7783ed5fdb4a43e87acdef7fa71955eb05b1a88: Status 404 returned error can't find the container with id 039fe7dcc8471eb18319a1b4e7783ed5fdb4a43e87acdef7fa71955eb05b1a88 Jul 22 19:01:31.048738 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:31.048707 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" event={"ID":"8c38839a-dab6-4cbe-b004-641a60a1138a","Type":"ContainerStarted","Data":"039fe7dcc8471eb18319a1b4e7783ed5fdb4a43e87acdef7fa71955eb05b1a88"} Jul 22 19:01:33.059282 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:33.059246 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" event={"ID":"8c38839a-dab6-4cbe-b004-641a60a1138a","Type":"ContainerStarted","Data":"5805a678751f9319b8326b1188a30ac01ccefa4755def1602edb029a204ed064"} Jul 22 19:01:33.059716 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:33.059337 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:33.081693 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:33.081652 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" podStartSLOduration=2.052729603 podStartE2EDuration="4.081638413s" totalImagesPullingTime="2.02890881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:33.080336617 +0000 UTC m=+626.742910975" watchObservedRunningTime="2026-07-22 19:01:33.081638413 +0000 UTC m=+626.744212751" Jul 22 19:01:40.632969 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.632936 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh"] Jul 22 19:01:40.635495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.635471 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.639330 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.639308 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 19:01:40.639434 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.639308 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 19:01:40.639631 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.639617 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 19:01:40.639783 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.639764 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-jzqdn\"" Jul 22 19:01:40.656500 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.656469 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh"] Jul 22 19:01:40.665534 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.665487 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.665657 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.665553 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.665657 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.665627 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.665828 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.665698 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.665828 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.665758 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.665828 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.665813 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d94nn\" (UniqueName: \"kubernetes.io/projected/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-kube-api-access-d94nn\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.665983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.665863 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.665983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.665893 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.665983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.665957 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.767036 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.766995 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d94nn\" (UniqueName: \"kubernetes.io/projected/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-kube-api-access-d94nn\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.767202 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.767125 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.767202 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.767177 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.767289 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.767240 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.767289 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.767279 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.767398 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.767306 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.767398 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.767338 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.767398 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.767367 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.767562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.767417 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.767840 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.767815 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.768229 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.768206 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.768316 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.768268 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.768377 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.768348 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.768497 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.768469 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.769854 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.769830 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.769933 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.769892 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.776677 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.776649 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.776836 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.776815 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d94nn\" (UniqueName: \"kubernetes.io/projected/a1fe132e-14d8-4ff6-bf56-c0285356b9f6-kube-api-access-d94nn\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4fp5fh\" (UID: \"a1fe132e-14d8-4ff6-bf56-c0285356b9f6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:40.946259 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:40.946183 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:41.086543 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.086473 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh"] Jul 22 19:01:41.088807 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:01:41.088774 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1fe132e_14d8_4ff6_bf56_c0285356b9f6.slice/crio-4e1bd585780f14b7dc96ab147adff1a0a11ebae80249287a8813c191b438f59a WatchSource:0}: Error finding container 4e1bd585780f14b7dc96ab147adff1a0a11ebae80249287a8813c191b438f59a: Status 404 returned error can't find the container with id 4e1bd585780f14b7dc96ab147adff1a0a11ebae80249287a8813c191b438f59a Jul 22 19:01:41.095183 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.095150 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" event={"ID":"a1fe132e-14d8-4ff6-bf56-c0285356b9f6","Type":"ContainerStarted","Data":"4e1bd585780f14b7dc96ab147adff1a0a11ebae80249287a8813c191b438f59a"} Jul 22 19:01:41.191679 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.191648 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4"] Jul 22 19:01:41.194238 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.194218 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.202183 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.197546 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 19:01:41.202805 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.202774 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 19:01:41.202805 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.202790 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 19:01:41.202943 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.202807 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xjlgz\"" Jul 22 19:01:41.212344 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.212321 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4"] Jul 22 19:01:41.271797 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.271761 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8637487b-71d7-44e0-a324-ade94750d656-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.271954 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.271798 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xqhb\" (UniqueName: \"kubernetes.io/projected/8637487b-71d7-44e0-a324-ade94750d656-kube-api-access-2xqhb\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.271954 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.271834 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8637487b-71d7-44e0-a324-ade94750d656-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.271954 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.271938 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8637487b-71d7-44e0-a324-ade94750d656-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.272083 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.271965 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8637487b-71d7-44e0-a324-ade94750d656-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.272083 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.271984 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8637487b-71d7-44e0-a324-ade94750d656-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.272083 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.272010 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8637487b-71d7-44e0-a324-ade94750d656-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.373125 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.373096 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8637487b-71d7-44e0-a324-ade94750d656-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.373125 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.373129 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8637487b-71d7-44e0-a324-ade94750d656-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.373348 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.373145 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8637487b-71d7-44e0-a324-ade94750d656-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.373348 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.373169 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8637487b-71d7-44e0-a324-ade94750d656-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.373348 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.373188 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8637487b-71d7-44e0-a324-ade94750d656-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.373348 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.373210 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xqhb\" (UniqueName: \"kubernetes.io/projected/8637487b-71d7-44e0-a324-ade94750d656-kube-api-access-2xqhb\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.373348 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.373237 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8637487b-71d7-44e0-a324-ade94750d656-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.374421 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.374388 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8637487b-71d7-44e0-a324-ade94750d656-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.375805 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.375779 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8637487b-71d7-44e0-a324-ade94750d656-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.375940 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.375921 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8637487b-71d7-44e0-a324-ade94750d656-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.376017 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.375966 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8637487b-71d7-44e0-a324-ade94750d656-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.376081 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.376052 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8637487b-71d7-44e0-a324-ade94750d656-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.381464 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.381446 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8637487b-71d7-44e0-a324-ade94750d656-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.381832 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.381815 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xqhb\" (UniqueName: \"kubernetes.io/projected/8637487b-71d7-44e0-a324-ade94750d656-kube-api-access-2xqhb\") pod \"istiod-openshift-gateway-94bb8fbfd-n57v4\" (UID: \"8637487b-71d7-44e0-a324-ade94750d656\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.509113 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.509013 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:41.644399 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:41.644376 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4"] Jul 22 19:01:41.646313 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:01:41.646289 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8637487b_71d7_44e0_a324_ade94750d656.slice/crio-41534804f73b3201e62088f91a8790069cf3bd2de91369aa0a5ea12a2a1ff546 WatchSource:0}: Error finding container 41534804f73b3201e62088f91a8790069cf3bd2de91369aa0a5ea12a2a1ff546: Status 404 returned error can't find the container with id 41534804f73b3201e62088f91a8790069cf3bd2de91369aa0a5ea12a2a1ff546 Jul 22 19:01:42.100764 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:42.100735 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" event={"ID":"8637487b-71d7-44e0-a324-ade94750d656","Type":"ContainerStarted","Data":"41534804f73b3201e62088f91a8790069cf3bd2de91369aa0a5ea12a2a1ff546"} Jul 22 19:01:44.066189 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:44.066156 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c9874fb98-gt9tk" Jul 22 19:01:45.561300 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:45.561255 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:01:45.561640 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:45.561340 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:01:45.561640 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:45.561378 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:01:45.566032 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:45.566002 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:01:45.566109 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:45.566059 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:01:46.127284 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:46.127161 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" event={"ID":"8637487b-71d7-44e0-a324-ade94750d656","Type":"ContainerStarted","Data":"cbcefb7244bedb0f361e058f538c9f17ca4077e60f8fb7073ba6e977e9a645c6"} Jul 22 19:01:46.128199 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:46.128169 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:46.133490 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:46.133459 2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-n57v4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:01:46.133648 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:46.133533 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" podUID="8637487b-71d7-44e0-a324-ade94750d656" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:01:46.133648 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:46.133592 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" event={"ID":"a1fe132e-14d8-4ff6-bf56-c0285356b9f6","Type":"ContainerStarted","Data":"858404f307f116793a8bd3300c147d0e5e72004bda32df07d60f560286120f94"} Jul 22 19:01:46.154973 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:46.154916 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" podStartSLOduration=1.237504768 podStartE2EDuration="5.154900589s" totalImagesPullingTime="3.917395821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:46.152650535 +0000 UTC m=+639.815224875" watchObservedRunningTime="2026-07-22 19:01:46.154900589 +0000 UTC m=+639.817474936" Jul 22 19:01:46.187959 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:46.187900 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" podStartSLOduration=1.717787109 podStartE2EDuration="6.187879032s" totalImagesPullingTime="4.470091923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:01:46.186953804 +0000 UTC m=+639.849528143" watchObservedRunningTime="2026-07-22 19:01:46.187879032 +0000 UTC m=+639.850453374" Jul 22 19:01:46.946846 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:46.946815 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:46.951277 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:46.951254 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:47.140046 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:47.140011 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:01:47.140640 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:47.140617 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-n57v4" Jul 22 19:01:47.140986 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:01:47.140966 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4fp5fh" Jul 22 19:02:36.417124 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.417083 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78448657d6-wqtq6"] Jul 22 19:02:36.419801 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.419774 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.423158 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.423128 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-trusted-ca-bundle\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.423268 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.423191 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/495e2b8c-43c5-49e4-b170-f7ce121d26cd-console-oauth-config\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.423268 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.423226 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-oauth-serving-cert\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.423268 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.423262 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz2m9\" (UniqueName: \"kubernetes.io/projected/495e2b8c-43c5-49e4-b170-f7ce121d26cd-kube-api-access-tz2m9\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.423394 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.423327 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-console-config\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.423394 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.423358 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/495e2b8c-43c5-49e4-b170-f7ce121d26cd-console-serving-cert\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.423394 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.423389 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-service-ca\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.432946 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.432919 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78448657d6-wqtq6"] Jul 22 19:02:36.523869 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.523841 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-trusted-ca-bundle\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.523869 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.523875 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/495e2b8c-43c5-49e4-b170-f7ce121d26cd-console-oauth-config\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.524208 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.523902 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-oauth-serving-cert\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.524208 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.523932 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz2m9\" (UniqueName: \"kubernetes.io/projected/495e2b8c-43c5-49e4-b170-f7ce121d26cd-kube-api-access-tz2m9\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.524208 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.523971 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-console-config\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.524208 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.524003 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/495e2b8c-43c5-49e4-b170-f7ce121d26cd-console-serving-cert\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.524208 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.524034 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-service-ca\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.524820 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.524793 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-oauth-serving-cert\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.524928 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.524793 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-console-config\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.524928 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.524828 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-trusted-ca-bundle\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.525051 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.525029 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/495e2b8c-43c5-49e4-b170-f7ce121d26cd-service-ca\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.526486 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.526465 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/495e2b8c-43c5-49e4-b170-f7ce121d26cd-console-oauth-config\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.526660 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.526645 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/495e2b8c-43c5-49e4-b170-f7ce121d26cd-console-serving-cert\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.532091 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.532068 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz2m9\" (UniqueName: \"kubernetes.io/projected/495e2b8c-43c5-49e4-b170-f7ce121d26cd-kube-api-access-tz2m9\") pod \"console-78448657d6-wqtq6\" (UID: \"495e2b8c-43c5-49e4-b170-f7ce121d26cd\") " pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.730227 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.730131 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:36.866542 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:36.866499 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78448657d6-wqtq6"] Jul 22 19:02:36.868023 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:02:36.867988 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod495e2b8c_43c5_49e4_b170_f7ce121d26cd.slice/crio-1de338ee93eef5f310f9a5e916afdd3f59729e44418b1a527bcda83e8ae07393 WatchSource:0}: Error finding container 1de338ee93eef5f310f9a5e916afdd3f59729e44418b1a527bcda83e8ae07393: Status 404 returned error can't find the container with id 1de338ee93eef5f310f9a5e916afdd3f59729e44418b1a527bcda83e8ae07393 Jul 22 19:02:37.367315 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:37.367274 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78448657d6-wqtq6" event={"ID":"495e2b8c-43c5-49e4-b170-f7ce121d26cd","Type":"ContainerStarted","Data":"3c692db438c971b6c6ad72fe416494edce051eff4d3deb09a9a331f8a460234e"} Jul 22 19:02:37.367485 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:37.367337 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78448657d6-wqtq6" event={"ID":"495e2b8c-43c5-49e4-b170-f7ce121d26cd","Type":"ContainerStarted","Data":"1de338ee93eef5f310f9a5e916afdd3f59729e44418b1a527bcda83e8ae07393"} Jul 22 19:02:37.384399 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:37.384349 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78448657d6-wqtq6" podStartSLOduration=1.384332662 podStartE2EDuration="1.384332662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:02:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:02:37.383662654 +0000 UTC m=+691.046236993" watchObservedRunningTime="2026-07-22 19:02:37.384332662 +0000 UTC m=+691.046906995" Jul 22 19:02:46.730393 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:46.730356 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:46.730906 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:46.730425 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:46.734819 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:46.734796 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:47.418275 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:47.418244 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78448657d6-wqtq6" Jul 22 19:02:47.463144 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:47.463094 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c694df7f9-x82bd"] Jul 22 19:02:49.575297 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.575268 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt"] Jul 22 19:02:49.577767 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.577751 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:49.580488 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.580471 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 19:02:49.580609 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.580551 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 19:02:49.581682 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.581667 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cgr69\"" Jul 22 19:02:49.586390 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.586371 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt"] Jul 22 19:02:49.632362 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.632335 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:49.632470 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.632370 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c629\" (UniqueName: \"kubernetes.io/projected/dcae49ee-0270-44ff-bf97-67ef5d318210-kube-api-access-2c629\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:49.632470 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.632393 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:49.733250 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.733222 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:49.733365 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.733268 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c629\" (UniqueName: \"kubernetes.io/projected/dcae49ee-0270-44ff-bf97-67ef5d318210-kube-api-access-2c629\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:49.733365 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.733303 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:49.733617 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.733589 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:49.733706 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.733677 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:49.741705 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.741686 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c629\" (UniqueName: \"kubernetes.io/projected/dcae49ee-0270-44ff-bf97-67ef5d318210-kube-api-access-2c629\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:49.888220 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:49.888190 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:50.050488 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.050464 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt"] Jul 22 19:02:50.052805 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:02:50.052774 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcae49ee_0270_44ff_bf97_67ef5d318210.slice/crio-7507a5a66d62c63fc7f5b61d9e336b34be0e7f170b79eafaf889cd52469b17f1 WatchSource:0}: Error finding container 7507a5a66d62c63fc7f5b61d9e336b34be0e7f170b79eafaf889cd52469b17f1: Status 404 returned error can't find the container with id 7507a5a66d62c63fc7f5b61d9e336b34be0e7f170b79eafaf889cd52469b17f1 Jul 22 19:02:50.170539 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.170493 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw"] Jul 22 19:02:50.173336 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.173305 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.181314 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.181289 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw"] Jul 22 19:02:50.238040 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.238011 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg2qf\" (UniqueName: \"kubernetes.io/projected/f94d5404-cebf-4372-b51c-f0be34e2df6a-kube-api-access-cg2qf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.238139 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.238057 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.238139 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.238075 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.338798 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.338768 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.338938 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.338809 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.338938 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.338916 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg2qf\" (UniqueName: \"kubernetes.io/projected/f94d5404-cebf-4372-b51c-f0be34e2df6a-kube-api-access-cg2qf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.339119 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.339098 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.339182 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.339142 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.347160 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.347135 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg2qf\" (UniqueName: \"kubernetes.io/projected/f94d5404-cebf-4372-b51c-f0be34e2df6a-kube-api-access-cg2qf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.430270 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.430212 2584 generic.go:356] "Generic (PLEG): container finished" podID="dcae49ee-0270-44ff-bf97-67ef5d318210" containerID="974e87346c491abd65287934a254affa95ac214a2149342f9e0a8f93540bb936" exitCode=0 Jul 22 19:02:50.430357 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.430303 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" event={"ID":"dcae49ee-0270-44ff-bf97-67ef5d318210","Type":"ContainerDied","Data":"974e87346c491abd65287934a254affa95ac214a2149342f9e0a8f93540bb936"} Jul 22 19:02:50.430408 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.430358 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" event={"ID":"dcae49ee-0270-44ff-bf97-67ef5d318210","Type":"ContainerStarted","Data":"7507a5a66d62c63fc7f5b61d9e336b34be0e7f170b79eafaf889cd52469b17f1"} Jul 22 19:02:50.506609 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.506584 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:50.637577 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.637550 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw"] Jul 22 19:02:50.639688 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:02:50.639661 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94d5404_cebf_4372_b51c_f0be34e2df6a.slice/crio-79a27aadeb45502bafdb26a0eac79fae31aa5f7770d4b41196c459ab5ec46ee1 WatchSource:0}: Error finding container 79a27aadeb45502bafdb26a0eac79fae31aa5f7770d4b41196c459ab5ec46ee1: Status 404 returned error can't find the container with id 79a27aadeb45502bafdb26a0eac79fae31aa5f7770d4b41196c459ab5ec46ee1 Jul 22 19:02:50.775496 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.775466 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv"] Jul 22 19:02:50.777670 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.777654 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:50.788458 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.788429 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv"] Jul 22 19:02:50.843306 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.843282 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:50.843426 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.843326 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:50.843426 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.843387 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv8js\" (UniqueName: \"kubernetes.io/projected/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-kube-api-access-dv8js\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:50.944067 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.943995 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:50.944067 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.944046 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:50.944232 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.944073 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv8js\" (UniqueName: \"kubernetes.io/projected/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-kube-api-access-dv8js\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:50.944412 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.944394 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:50.944449 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.944391 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:50.952729 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:50.952701 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv8js\" (UniqueName: \"kubernetes.io/projected/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-kube-api-access-dv8js\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:51.101240 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.101219 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:51.178186 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.178154 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf"] Jul 22 19:02:51.180380 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.180358 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.192759 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.192724 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf"] Jul 22 19:02:51.239584 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.239503 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv"] Jul 22 19:02:51.241529 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:02:51.241486 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dca680d_25d4_41e9_b2bd_7aa4cf63b0af.slice/crio-7aaf94cfd80cc97aeb3af7885e55c5a601a323c91dd19313abaaaa74e78c3265 WatchSource:0}: Error finding container 7aaf94cfd80cc97aeb3af7885e55c5a601a323c91dd19313abaaaa74e78c3265: Status 404 returned error can't find the container with id 7aaf94cfd80cc97aeb3af7885e55c5a601a323c91dd19313abaaaa74e78c3265 Jul 22 19:02:51.247363 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.247338 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.247455 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.247370 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvfrk\" (UniqueName: \"kubernetes.io/projected/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-kube-api-access-mvfrk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.247502 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.247459 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.348013 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.347987 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.348172 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.348021 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvfrk\" (UniqueName: \"kubernetes.io/projected/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-kube-api-access-mvfrk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.348172 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.348083 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.348341 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.348319 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.348407 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.348349 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.356267 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.356247 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvfrk\" (UniqueName: \"kubernetes.io/projected/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-kube-api-access-mvfrk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.436905 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.436874 2584 generic.go:356] "Generic (PLEG): container finished" podID="f94d5404-cebf-4372-b51c-f0be34e2df6a" containerID="9e412d88cddf293d0f6c2e90e1005148709815a0d47955c5e728d64857053962" exitCode=0 Jul 22 19:02:51.437087 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.436924 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" event={"ID":"f94d5404-cebf-4372-b51c-f0be34e2df6a","Type":"ContainerDied","Data":"9e412d88cddf293d0f6c2e90e1005148709815a0d47955c5e728d64857053962"} Jul 22 19:02:51.437087 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.436973 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" event={"ID":"f94d5404-cebf-4372-b51c-f0be34e2df6a","Type":"ContainerStarted","Data":"79a27aadeb45502bafdb26a0eac79fae31aa5f7770d4b41196c459ab5ec46ee1"} Jul 22 19:02:51.438811 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.438792 2584 generic.go:356] "Generic (PLEG): container finished" podID="dcae49ee-0270-44ff-bf97-67ef5d318210" containerID="ed231c796299e1796bbae6fd35fed7b3412787b45e5d32afb6e0725d77700ec2" exitCode=0 Jul 22 19:02:51.438919 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.438873 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" event={"ID":"dcae49ee-0270-44ff-bf97-67ef5d318210","Type":"ContainerDied","Data":"ed231c796299e1796bbae6fd35fed7b3412787b45e5d32afb6e0725d77700ec2"} Jul 22 19:02:51.440544 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.440464 2584 generic.go:356] "Generic (PLEG): container finished" podID="2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" containerID="c3898074700c8ea8db53aa067c52d43d4af0a3c990f889b34ab4ca7ed07ce9a1" exitCode=0 Jul 22 19:02:51.440635 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.440537 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" event={"ID":"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af","Type":"ContainerDied","Data":"c3898074700c8ea8db53aa067c52d43d4af0a3c990f889b34ab4ca7ed07ce9a1"} Jul 22 19:02:51.440635 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.440584 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" event={"ID":"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af","Type":"ContainerStarted","Data":"7aaf94cfd80cc97aeb3af7885e55c5a601a323c91dd19313abaaaa74e78c3265"} Jul 22 19:02:51.501630 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.501604 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:51.646778 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:51.646749 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf"] Jul 22 19:02:51.648456 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:02:51.648436 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod410de48d_1fe6_4dde_bdd1_35d9d05cd0fa.slice/crio-98038d0027cbcc78510a9c5f609796be5d502bf6b59f9f05cdfa81459079eb6b WatchSource:0}: Error finding container 98038d0027cbcc78510a9c5f609796be5d502bf6b59f9f05cdfa81459079eb6b: Status 404 returned error can't find the container with id 98038d0027cbcc78510a9c5f609796be5d502bf6b59f9f05cdfa81459079eb6b Jul 22 19:02:52.447736 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:52.447710 2584 generic.go:356] "Generic (PLEG): container finished" podID="f94d5404-cebf-4372-b51c-f0be34e2df6a" containerID="55fe1f74bc139bb9dcd7ef6c20719c4a34b47d6473f239121bffca63dc208cb1" exitCode=0 Jul 22 19:02:52.447840 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:52.447786 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" event={"ID":"f94d5404-cebf-4372-b51c-f0be34e2df6a","Type":"ContainerDied","Data":"55fe1f74bc139bb9dcd7ef6c20719c4a34b47d6473f239121bffca63dc208cb1"} Jul 22 19:02:52.449931 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:52.449909 2584 generic.go:356] "Generic (PLEG): container finished" podID="dcae49ee-0270-44ff-bf97-67ef5d318210" containerID="0abea94948905c52ad139e5456616801e361051108e05015e0a39b38b5f2b901" exitCode=0 Jul 22 19:02:52.450034 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:52.449983 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" event={"ID":"dcae49ee-0270-44ff-bf97-67ef5d318210","Type":"ContainerDied","Data":"0abea94948905c52ad139e5456616801e361051108e05015e0a39b38b5f2b901"} Jul 22 19:02:52.451360 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:52.451341 2584 generic.go:356] "Generic (PLEG): container finished" podID="410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" containerID="50cd78791046d44d7c12c5e6970478a56539c37392af743613ffcf7f6af5869b" exitCode=0 Jul 22 19:02:52.451475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:52.451363 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" event={"ID":"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa","Type":"ContainerDied","Data":"50cd78791046d44d7c12c5e6970478a56539c37392af743613ffcf7f6af5869b"} Jul 22 19:02:52.451475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:52.451405 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" event={"ID":"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa","Type":"ContainerStarted","Data":"98038d0027cbcc78510a9c5f609796be5d502bf6b59f9f05cdfa81459079eb6b"} Jul 22 19:02:52.453278 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:52.453254 2584 generic.go:356] "Generic (PLEG): container finished" podID="2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" containerID="850472ad352d16ed7201e2a1ce419b8045332c95112520d945a3925b939f6d31" exitCode=0 Jul 22 19:02:52.453399 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:52.453303 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" event={"ID":"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af","Type":"ContainerDied","Data":"850472ad352d16ed7201e2a1ce419b8045332c95112520d945a3925b939f6d31"} Jul 22 19:02:53.460403 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.460309 2584 generic.go:356] "Generic (PLEG): container finished" podID="410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" containerID="77def0ca417a88c2f1de2361229968c43efb02e3f9bd71ff14dfc234d788afb1" exitCode=0 Jul 22 19:02:53.460821 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.460396 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" event={"ID":"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa","Type":"ContainerDied","Data":"77def0ca417a88c2f1de2361229968c43efb02e3f9bd71ff14dfc234d788afb1"} Jul 22 19:02:53.462723 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.462700 2584 generic.go:356] "Generic (PLEG): container finished" podID="2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" containerID="c4694dcb1a7e0654b2fd816bbe926c3dde63db8430cc4429789b6674a4dd1bdb" exitCode=0 Jul 22 19:02:53.462843 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.462780 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" event={"ID":"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af","Type":"ContainerDied","Data":"c4694dcb1a7e0654b2fd816bbe926c3dde63db8430cc4429789b6674a4dd1bdb"} Jul 22 19:02:53.464864 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.464837 2584 generic.go:356] "Generic (PLEG): container finished" podID="f94d5404-cebf-4372-b51c-f0be34e2df6a" containerID="d2e92d214866b22e44713f14f4d4f0cd842d305469320073d846294f878c1dd0" exitCode=0 Jul 22 19:02:53.464934 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.464863 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" event={"ID":"f94d5404-cebf-4372-b51c-f0be34e2df6a","Type":"ContainerDied","Data":"d2e92d214866b22e44713f14f4d4f0cd842d305469320073d846294f878c1dd0"} Jul 22 19:02:53.660445 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.660421 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:53.767047 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.766963 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c629\" (UniqueName: \"kubernetes.io/projected/dcae49ee-0270-44ff-bf97-67ef5d318210-kube-api-access-2c629\") pod \"dcae49ee-0270-44ff-bf97-67ef5d318210\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " Jul 22 19:02:53.767047 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.767013 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-bundle\") pod \"dcae49ee-0270-44ff-bf97-67ef5d318210\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " Jul 22 19:02:53.767047 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.767047 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-util\") pod \"dcae49ee-0270-44ff-bf97-67ef5d318210\" (UID: \"dcae49ee-0270-44ff-bf97-67ef5d318210\") " Jul 22 19:02:53.767683 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.767649 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-bundle" (OuterVolumeSpecName: "bundle") pod "dcae49ee-0270-44ff-bf97-67ef5d318210" (UID: "dcae49ee-0270-44ff-bf97-67ef5d318210"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:02:53.769189 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.769168 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcae49ee-0270-44ff-bf97-67ef5d318210-kube-api-access-2c629" (OuterVolumeSpecName: "kube-api-access-2c629") pod "dcae49ee-0270-44ff-bf97-67ef5d318210" (UID: "dcae49ee-0270-44ff-bf97-67ef5d318210"). InnerVolumeSpecName "kube-api-access-2c629". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:02:53.772028 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.772010 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-util" (OuterVolumeSpecName: "util") pod "dcae49ee-0270-44ff-bf97-67ef5d318210" (UID: "dcae49ee-0270-44ff-bf97-67ef5d318210"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:02:53.867600 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.867573 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c629\" (UniqueName: \"kubernetes.io/projected/dcae49ee-0270-44ff-bf97-67ef5d318210-kube-api-access-2c629\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:53.867600 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.867600 2584 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-bundle\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:53.867727 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:53.867613 2584 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcae49ee-0270-44ff-bf97-67ef5d318210-util\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:54.472235 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.472208 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" Jul 22 19:02:54.472665 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.472203 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt" event={"ID":"dcae49ee-0270-44ff-bf97-67ef5d318210","Type":"ContainerDied","Data":"7507a5a66d62c63fc7f5b61d9e336b34be0e7f170b79eafaf889cd52469b17f1"} Jul 22 19:02:54.472665 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.472317 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7507a5a66d62c63fc7f5b61d9e336b34be0e7f170b79eafaf889cd52469b17f1" Jul 22 19:02:54.474437 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.474411 2584 generic.go:356] "Generic (PLEG): container finished" podID="410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" containerID="d8d20e2045f9b90d28fbd51b524a313f3394134102c1f168f4101be3e4ed14ee" exitCode=0 Jul 22 19:02:54.474592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.474489 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" event={"ID":"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa","Type":"ContainerDied","Data":"d8d20e2045f9b90d28fbd51b524a313f3394134102c1f168f4101be3e4ed14ee"} Jul 22 19:02:54.631480 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.631457 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:54.634124 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.634105 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:54.674422 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.674396 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-util\") pod \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " Jul 22 19:02:54.674590 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.674438 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg2qf\" (UniqueName: \"kubernetes.io/projected/f94d5404-cebf-4372-b51c-f0be34e2df6a-kube-api-access-cg2qf\") pod \"f94d5404-cebf-4372-b51c-f0be34e2df6a\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " Jul 22 19:02:54.674590 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.674464 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dv8js\" (UniqueName: \"kubernetes.io/projected/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-kube-api-access-dv8js\") pod \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " Jul 22 19:02:54.674590 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.674490 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-util\") pod \"f94d5404-cebf-4372-b51c-f0be34e2df6a\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " Jul 22 19:02:54.674775 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.674597 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-bundle\") pod \"f94d5404-cebf-4372-b51c-f0be34e2df6a\" (UID: \"f94d5404-cebf-4372-b51c-f0be34e2df6a\") " Jul 22 19:02:54.674775 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.674650 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-bundle\") pod \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\" (UID: \"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af\") " Jul 22 19:02:54.675830 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.675391 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-bundle" (OuterVolumeSpecName: "bundle") pod "f94d5404-cebf-4372-b51c-f0be34e2df6a" (UID: "f94d5404-cebf-4372-b51c-f0be34e2df6a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:02:54.675830 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.675589 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-bundle" (OuterVolumeSpecName: "bundle") pod "2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" (UID: "2dca680d-25d4-41e9-b2bd-7aa4cf63b0af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:02:54.677011 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.676975 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f94d5404-cebf-4372-b51c-f0be34e2df6a-kube-api-access-cg2qf" (OuterVolumeSpecName: "kube-api-access-cg2qf") pod "f94d5404-cebf-4372-b51c-f0be34e2df6a" (UID: "f94d5404-cebf-4372-b51c-f0be34e2df6a"). InnerVolumeSpecName "kube-api-access-cg2qf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:02:54.677011 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.676970 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-kube-api-access-dv8js" (OuterVolumeSpecName: "kube-api-access-dv8js") pod "2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" (UID: "2dca680d-25d4-41e9-b2bd-7aa4cf63b0af"). InnerVolumeSpecName "kube-api-access-dv8js". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:02:54.681305 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.681276 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-util" (OuterVolumeSpecName: "util") pod "2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" (UID: "2dca680d-25d4-41e9-b2bd-7aa4cf63b0af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:02:54.683744 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.683710 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-util" (OuterVolumeSpecName: "util") pod "f94d5404-cebf-4372-b51c-f0be34e2df6a" (UID: "f94d5404-cebf-4372-b51c-f0be34e2df6a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:02:54.775551 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.775444 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg2qf\" (UniqueName: \"kubernetes.io/projected/f94d5404-cebf-4372-b51c-f0be34e2df6a-kube-api-access-cg2qf\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:54.775551 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.775473 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv8js\" (UniqueName: \"kubernetes.io/projected/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-kube-api-access-dv8js\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:54.775551 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.775483 2584 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-util\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:54.775551 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.775493 2584 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f94d5404-cebf-4372-b51c-f0be34e2df6a-bundle\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:54.775551 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.775501 2584 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-bundle\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:54.775551 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:54.775508 2584 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dca680d-25d4-41e9-b2bd-7aa4cf63b0af-util\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:55.481302 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.481264 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" Jul 22 19:02:55.481786 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.481272 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv" event={"ID":"2dca680d-25d4-41e9-b2bd-7aa4cf63b0af","Type":"ContainerDied","Data":"7aaf94cfd80cc97aeb3af7885e55c5a601a323c91dd19313abaaaa74e78c3265"} Jul 22 19:02:55.481786 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.481396 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7aaf94cfd80cc97aeb3af7885e55c5a601a323c91dd19313abaaaa74e78c3265" Jul 22 19:02:55.483211 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.483179 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" event={"ID":"f94d5404-cebf-4372-b51c-f0be34e2df6a","Type":"ContainerDied","Data":"79a27aadeb45502bafdb26a0eac79fae31aa5f7770d4b41196c459ab5ec46ee1"} Jul 22 19:02:55.483370 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.483229 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79a27aadeb45502bafdb26a0eac79fae31aa5f7770d4b41196c459ab5ec46ee1" Jul 22 19:02:55.483439 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.483399 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw" Jul 22 19:02:55.615950 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.615926 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:55.683754 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.683721 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvfrk\" (UniqueName: \"kubernetes.io/projected/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-kube-api-access-mvfrk\") pod \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " Jul 22 19:02:55.683916 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.683781 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-util\") pod \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " Jul 22 19:02:55.683916 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.683847 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-bundle\") pod \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\" (UID: \"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa\") " Jul 22 19:02:55.684348 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.684311 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-bundle" (OuterVolumeSpecName: "bundle") pod "410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" (UID: "410de48d-1fe6-4dde-bdd1-35d9d05cd0fa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:02:55.686415 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.686383 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-kube-api-access-mvfrk" (OuterVolumeSpecName: "kube-api-access-mvfrk") pod "410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" (UID: "410de48d-1fe6-4dde-bdd1-35d9d05cd0fa"). InnerVolumeSpecName "kube-api-access-mvfrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:02:55.690129 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.690108 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-util" (OuterVolumeSpecName: "util") pod "410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" (UID: "410de48d-1fe6-4dde-bdd1-35d9d05cd0fa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:02:55.784867 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.784784 2584 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-util\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:55.784867 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.784814 2584 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-bundle\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:55.784867 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:55.784824 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvfrk\" (UniqueName: \"kubernetes.io/projected/410de48d-1fe6-4dde-bdd1-35d9d05cd0fa-kube-api-access-mvfrk\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:02:56.490014 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:56.489979 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" Jul 22 19:02:56.490014 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:56.489997 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf" event={"ID":"410de48d-1fe6-4dde-bdd1-35d9d05cd0fa","Type":"ContainerDied","Data":"98038d0027cbcc78510a9c5f609796be5d502bf6b59f9f05cdfa81459079eb6b"} Jul 22 19:02:56.490421 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:02:56.490037 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98038d0027cbcc78510a9c5f609796be5d502bf6b59f9f05cdfa81459079eb6b" Jul 22 19:03:12.486550 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.486489 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c694df7f9-x82bd" podUID="b99d8976-f943-4a5f-8afa-6c00a41ee1e4" containerName="console" containerID="cri-o://6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8" gracePeriod=15 Jul 22 19:03:12.743231 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.743181 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 19:03:12.833837 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.833807 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-trusted-ca-bundle\") pod \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " Jul 22 19:03:12.833990 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.833839 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-oauth-config\") pod \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " Jul 22 19:03:12.833990 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.833865 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtm9c\" (UniqueName: \"kubernetes.io/projected/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-kube-api-access-wtm9c\") pod \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " Jul 22 19:03:12.833990 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.833918 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-serving-cert\") pod \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " Jul 22 19:03:12.833990 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.833957 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-oauth-serving-cert\") pod \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " Jul 22 19:03:12.834203 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.834129 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-service-ca\") pod \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " Jul 22 19:03:12.834258 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.834229 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-config\") pod \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\" (UID: \"b99d8976-f943-4a5f-8afa-6c00a41ee1e4\") " Jul 22 19:03:12.834482 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.834384 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b99d8976-f943-4a5f-8afa-6c00a41ee1e4" (UID: "b99d8976-f943-4a5f-8afa-6c00a41ee1e4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:03:12.834632 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.834503 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-service-ca" (OuterVolumeSpecName: "service-ca") pod "b99d8976-f943-4a5f-8afa-6c00a41ee1e4" (UID: "b99d8976-f943-4a5f-8afa-6c00a41ee1e4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:03:12.834703 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.834626 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-config" (OuterVolumeSpecName: "console-config") pod "b99d8976-f943-4a5f-8afa-6c00a41ee1e4" (UID: "b99d8976-f943-4a5f-8afa-6c00a41ee1e4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:03:12.834754 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.834698 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b99d8976-f943-4a5f-8afa-6c00a41ee1e4" (UID: "b99d8976-f943-4a5f-8afa-6c00a41ee1e4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:03:12.834838 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.834822 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-oauth-serving-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:03:12.836250 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.836220 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b99d8976-f943-4a5f-8afa-6c00a41ee1e4" (UID: "b99d8976-f943-4a5f-8afa-6c00a41ee1e4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:03:12.836358 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.836266 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b99d8976-f943-4a5f-8afa-6c00a41ee1e4" (UID: "b99d8976-f943-4a5f-8afa-6c00a41ee1e4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:03:12.836358 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.836337 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-kube-api-access-wtm9c" (OuterVolumeSpecName: "kube-api-access-wtm9c") pod "b99d8976-f943-4a5f-8afa-6c00a41ee1e4" (UID: "b99d8976-f943-4a5f-8afa-6c00a41ee1e4"). InnerVolumeSpecName "kube-api-access-wtm9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:03:12.935438 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.935412 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-serving-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:03:12.935438 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.935437 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:03:12.935609 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.935446 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-config\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:03:12.935609 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.935454 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-trusted-ca-bundle\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:03:12.935609 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.935462 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-console-oauth-config\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:03:12.935609 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:12.935470 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtm9c\" (UniqueName: \"kubernetes.io/projected/b99d8976-f943-4a5f-8afa-6c00a41ee1e4-kube-api-access-wtm9c\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:03:13.574804 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:13.574771 2584 generic.go:356] "Generic (PLEG): container finished" podID="b99d8976-f943-4a5f-8afa-6c00a41ee1e4" containerID="6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8" exitCode=0 Jul 22 19:03:13.575270 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:13.574818 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c694df7f9-x82bd" Jul 22 19:03:13.575270 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:13.574866 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c694df7f9-x82bd" event={"ID":"b99d8976-f943-4a5f-8afa-6c00a41ee1e4","Type":"ContainerDied","Data":"6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8"} Jul 22 19:03:13.575270 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:13.574929 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c694df7f9-x82bd" event={"ID":"b99d8976-f943-4a5f-8afa-6c00a41ee1e4","Type":"ContainerDied","Data":"bd14caa8efa21e0d7412555bae1521e4f610397b3fb5129631dc5c70b9c72344"} Jul 22 19:03:13.575270 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:13.574961 2584 scope.go:117] "RemoveContainer" containerID="6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8" Jul 22 19:03:13.583847 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:13.583831 2584 scope.go:117] "RemoveContainer" containerID="6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8" Jul 22 19:03:13.584083 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:03:13.584067 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8\": container with ID starting with 6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8 not found: ID does not exist" containerID="6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8" Jul 22 19:03:13.584126 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:13.584089 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8"} err="failed to get container status \"6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8\": rpc error: code = NotFound desc = could not find container \"6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8\": container with ID starting with 6d06e0fd54a80f97b5770d99e1c2e2ec941b085bf52df8cb38257ef8cb283fc8 not found: ID does not exist" Jul 22 19:03:13.598618 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:13.598592 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c694df7f9-x82bd"] Jul 22 19:03:13.606699 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:13.606679 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c694df7f9-x82bd"] Jul 22 19:03:14.905828 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:03:14.905798 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b99d8976-f943-4a5f-8afa-6c00a41ee1e4" path="/var/lib/kubelet/pods/b99d8976-f943-4a5f-8afa-6c00a41ee1e4/volumes" Jul 22 19:04:10.185960 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.185917 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv"] Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186243 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcae49ee-0270-44ff-bf97-67ef5d318210" containerName="util" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186259 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcae49ee-0270-44ff-bf97-67ef5d318210" containerName="util" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186280 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b99d8976-f943-4a5f-8afa-6c00a41ee1e4" containerName="console" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186289 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b99d8976-f943-4a5f-8afa-6c00a41ee1e4" containerName="console" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186306 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94d5404-cebf-4372-b51c-f0be34e2df6a" containerName="pull" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186316 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94d5404-cebf-4372-b51c-f0be34e2df6a" containerName="pull" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186327 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" containerName="util" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186335 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" containerName="util" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186345 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" containerName="util" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186354 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" containerName="util" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186363 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" containerName="extract" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186372 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" containerName="extract" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186380 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" containerName="pull" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186388 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" containerName="pull" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186398 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcae49ee-0270-44ff-bf97-67ef5d318210" containerName="extract" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186406 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcae49ee-0270-44ff-bf97-67ef5d318210" containerName="extract" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186417 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94d5404-cebf-4372-b51c-f0be34e2df6a" containerName="extract" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186424 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94d5404-cebf-4372-b51c-f0be34e2df6a" containerName="extract" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186441 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94d5404-cebf-4372-b51c-f0be34e2df6a" containerName="util" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186449 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94d5404-cebf-4372-b51c-f0be34e2df6a" containerName="util" Jul 22 19:04:10.186447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186459 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcae49ee-0270-44ff-bf97-67ef5d318210" containerName="pull" Jul 22 19:04:10.187474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186467 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcae49ee-0270-44ff-bf97-67ef5d318210" containerName="pull" Jul 22 19:04:10.187474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186476 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" containerName="pull" Jul 22 19:04:10.187474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186483 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" containerName="pull" Jul 22 19:04:10.187474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186490 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" containerName="extract" Jul 22 19:04:10.187474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186495 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" containerName="extract" Jul 22 19:04:10.187474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186597 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dca680d-25d4-41e9-b2bd-7aa4cf63b0af" containerName="extract" Jul 22 19:04:10.187474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186606 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="410de48d-1fe6-4dde-bdd1-35d9d05cd0fa" containerName="extract" Jul 22 19:04:10.187474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186611 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b99d8976-f943-4a5f-8afa-6c00a41ee1e4" containerName="console" Jul 22 19:04:10.187474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186617 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f94d5404-cebf-4372-b51c-f0be34e2df6a" containerName="extract" Jul 22 19:04:10.187474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.186624 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcae49ee-0270-44ff-bf97-67ef5d318210" containerName="extract" Jul 22 19:04:10.190646 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.190625 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.193288 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.193268 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-94nkm\"" Jul 22 19:04:10.202731 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.202705 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv"] Jul 22 19:04:10.320782 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.320741 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3179ecc9-7c67-46ac-bb01-09336685a9a3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.320948 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.320800 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.320948 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.320831 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.320948 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.320889 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.320948 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.320936 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.321129 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.320963 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.321129 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.320991 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nlv4\" (UniqueName: \"kubernetes.io/projected/3179ecc9-7c67-46ac-bb01-09336685a9a3-kube-api-access-8nlv4\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.321129 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.321041 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.321129 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.321093 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.421688 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.421650 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.421862 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.421703 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.421862 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.421764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3179ecc9-7c67-46ac-bb01-09336685a9a3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.421862 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.421819 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.422027 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.421871 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.422027 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.421905 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.422027 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.421940 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.422027 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.421965 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.422027 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.421992 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nlv4\" (UniqueName: \"kubernetes.io/projected/3179ecc9-7c67-46ac-bb01-09336685a9a3-kube-api-access-8nlv4\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.422264 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.422086 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.422264 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.422244 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.422484 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.422459 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.422714 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.422689 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.422880 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.422861 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3179ecc9-7c67-46ac-bb01-09336685a9a3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.424859 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.424818 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.425189 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.425143 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.434282 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.434252 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3179ecc9-7c67-46ac-bb01-09336685a9a3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.434573 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.434550 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nlv4\" (UniqueName: \"kubernetes.io/projected/3179ecc9-7c67-46ac-bb01-09336685a9a3-kube-api-access-8nlv4\") pod \"maas-default-gateway-openshift-default-56654dcf7-7hwcv\" (UID: \"3179ecc9-7c67-46ac-bb01-09336685a9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.502065 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.501972 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:10.639050 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.639026 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv"] Jul 22 19:04:10.641231 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:04:10.641197 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3179ecc9_7c67_46ac_bb01_09336685a9a3.slice/crio-60c43d6cf80deec17d7a3e1f022721ee4a6926ad91af871a984041043a0e91d9 WatchSource:0}: Error finding container 60c43d6cf80deec17d7a3e1f022721ee4a6926ad91af871a984041043a0e91d9: Status 404 returned error can't find the container with id 60c43d6cf80deec17d7a3e1f022721ee4a6926ad91af871a984041043a0e91d9 Jul 22 19:04:10.647210 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.647169 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:04:10.647327 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.647263 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:04:10.647327 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.647308 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:04:10.856765 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.856727 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" event={"ID":"3179ecc9-7c67-46ac-bb01-09336685a9a3","Type":"ContainerStarted","Data":"9cab612d58b9d01aea4276923396b78aa7541d9ea8179685d412b84ec515bbca"} Jul 22 19:04:10.856971 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.856793 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" event={"ID":"3179ecc9-7c67-46ac-bb01-09336685a9a3","Type":"ContainerStarted","Data":"60c43d6cf80deec17d7a3e1f022721ee4a6926ad91af871a984041043a0e91d9"} Jul 22 19:04:10.882115 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:10.882050 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" podStartSLOduration=0.882028649 podStartE2EDuration="882.028649ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:04:10.88035825 +0000 UTC m=+784.542932586" watchObservedRunningTime="2026-07-22 19:04:10.882028649 +0000 UTC m=+784.544602989" Jul 22 19:04:11.502448 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:11.502405 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:11.507223 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:11.507200 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:11.862207 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:11.862174 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:04:11.863113 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:04:11.863094 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:05:33.414917 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.414880 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-569cdb7f56-xr224"] Jul 22 19:05:33.418167 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.418148 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:33.420909 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.420881 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 22 19:05:33.421028 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.420949 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 19:05:33.421028 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.420955 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 19:05:33.429710 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.429688 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-569cdb7f56-xr224"] Jul 22 19:05:33.511030 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.510990 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p7nc\" (UniqueName: \"kubernetes.io/projected/e54fe280-64a1-4210-ba2d-4df8469a74d5-kube-api-access-4p7nc\") pod \"maas-api-569cdb7f56-xr224\" (UID: \"e54fe280-64a1-4210-ba2d-4df8469a74d5\") " pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:33.511185 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.511051 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e54fe280-64a1-4210-ba2d-4df8469a74d5-maas-api-tls\") pod \"maas-api-569cdb7f56-xr224\" (UID: \"e54fe280-64a1-4210-ba2d-4df8469a74d5\") " pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:33.511185 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.511111 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e54fe280-64a1-4210-ba2d-4df8469a74d5-openshift-service-ca\") pod \"maas-api-569cdb7f56-xr224\" (UID: \"e54fe280-64a1-4210-ba2d-4df8469a74d5\") " pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:33.611769 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.611730 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p7nc\" (UniqueName: \"kubernetes.io/projected/e54fe280-64a1-4210-ba2d-4df8469a74d5-kube-api-access-4p7nc\") pod \"maas-api-569cdb7f56-xr224\" (UID: \"e54fe280-64a1-4210-ba2d-4df8469a74d5\") " pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:33.611946 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.611812 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e54fe280-64a1-4210-ba2d-4df8469a74d5-maas-api-tls\") pod \"maas-api-569cdb7f56-xr224\" (UID: \"e54fe280-64a1-4210-ba2d-4df8469a74d5\") " pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:33.611946 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.611869 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e54fe280-64a1-4210-ba2d-4df8469a74d5-openshift-service-ca\") pod \"maas-api-569cdb7f56-xr224\" (UID: \"e54fe280-64a1-4210-ba2d-4df8469a74d5\") " pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:33.612039 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:05:33.611955 2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 22 19:05:33.612088 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:05:33.612038 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e54fe280-64a1-4210-ba2d-4df8469a74d5-maas-api-tls podName:e54fe280-64a1-4210-ba2d-4df8469a74d5 nodeName:}" failed. No retries permitted until 2026-07-22 19:05:34.112018976 +0000 UTC m=+867.774593302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e54fe280-64a1-4210-ba2d-4df8469a74d5-maas-api-tls") pod "maas-api-569cdb7f56-xr224" (UID: "e54fe280-64a1-4210-ba2d-4df8469a74d5") : secret "maas-api-serving-cert" not found Jul 22 19:05:33.612638 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.612618 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e54fe280-64a1-4210-ba2d-4df8469a74d5-openshift-service-ca\") pod \"maas-api-569cdb7f56-xr224\" (UID: \"e54fe280-64a1-4210-ba2d-4df8469a74d5\") " pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:33.627000 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:33.626975 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p7nc\" (UniqueName: \"kubernetes.io/projected/e54fe280-64a1-4210-ba2d-4df8469a74d5-kube-api-access-4p7nc\") pod \"maas-api-569cdb7f56-xr224\" (UID: \"e54fe280-64a1-4210-ba2d-4df8469a74d5\") " pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:34.115315 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:34.115272 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e54fe280-64a1-4210-ba2d-4df8469a74d5-maas-api-tls\") pod \"maas-api-569cdb7f56-xr224\" (UID: \"e54fe280-64a1-4210-ba2d-4df8469a74d5\") " pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:34.117909 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:34.117886 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e54fe280-64a1-4210-ba2d-4df8469a74d5-maas-api-tls\") pod \"maas-api-569cdb7f56-xr224\" (UID: \"e54fe280-64a1-4210-ba2d-4df8469a74d5\") " pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:34.331160 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:34.331127 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:34.467024 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:34.466996 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-569cdb7f56-xr224"] Jul 22 19:05:34.469011 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:05:34.468978 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode54fe280_64a1_4210_ba2d_4df8469a74d5.slice/crio-3a957202ef4cd62d2c031023e8a5b427218bc154f9b09db89fb27768437655dc WatchSource:0}: Error finding container 3a957202ef4cd62d2c031023e8a5b427218bc154f9b09db89fb27768437655dc: Status 404 returned error can't find the container with id 3a957202ef4cd62d2c031023e8a5b427218bc154f9b09db89fb27768437655dc Jul 22 19:05:35.270905 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:35.270831 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" event={"ID":"e54fe280-64a1-4210-ba2d-4df8469a74d5","Type":"ContainerStarted","Data":"3a957202ef4cd62d2c031023e8a5b427218bc154f9b09db89fb27768437655dc"} Jul 22 19:05:38.289435 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:38.289391 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" event={"ID":"e54fe280-64a1-4210-ba2d-4df8469a74d5","Type":"ContainerStarted","Data":"555011ed937588fe5bd014dd309854c8a473a78f4fc4a154331bb7360559c2b5"} Jul 22 19:05:38.290225 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:38.290198 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:38.308408 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:38.308359 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" podStartSLOduration=2.468038722 podStartE2EDuration="5.308342851s" totalImagesPullingTime="2.840304129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:05:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:05:38.306734706 +0000 UTC m=+871.969309046" watchObservedRunningTime="2026-07-22 19:05:38.308342851 +0000 UTC m=+871.970917190" Jul 22 19:05:41.864179 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:41.864112 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7hwcv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 22 19:05:41.864590 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:41.864171 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" podUID="3179ecc9-7c67-46ac-bb01-09336685a9a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 22 19:05:42.311928 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:42.311893 2584 generic.go:356] "Generic (PLEG): container finished" podID="3179ecc9-7c67-46ac-bb01-09336685a9a3" containerID="9cab612d58b9d01aea4276923396b78aa7541d9ea8179685d412b84ec515bbca" exitCode=0 Jul 22 19:05:42.311928 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:42.311932 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" event={"ID":"3179ecc9-7c67-46ac-bb01-09336685a9a3","Type":"ContainerDied","Data":"9cab612d58b9d01aea4276923396b78aa7541d9ea8179685d412b84ec515bbca"} Jul 22 19:05:42.312343 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:42.312328 2584 scope.go:117] "RemoveContainer" containerID="9cab612d58b9d01aea4276923396b78aa7541d9ea8179685d412b84ec515bbca" Jul 22 19:05:42.313454 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:42.313424 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:05:42.313566 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:42.313509 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:05:42.313618 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:42.313594 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:05:43.321471 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:43.321413 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" event={"ID":"3179ecc9-7c67-46ac-bb01-09336685a9a3","Type":"ContainerStarted","Data":"1c02067698a207f67d40c3e7733022269821af209985e5d5854a0d44e5df5fbb"} Jul 22 19:05:43.503467 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:43.503422 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:05:43.504069 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:43.504037 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7hwcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 22 19:05:43.504165 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:43.504102 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" podUID="3179ecc9-7c67-46ac-bb01-09336685a9a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 22 19:05:44.503164 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:44.503116 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7hwcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 22 19:05:44.503661 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:44.503188 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" podUID="3179ecc9-7c67-46ac-bb01-09336685a9a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 22 19:05:45.308251 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:45.308215 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-569cdb7f56-xr224" Jul 22 19:05:45.502505 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:45.502458 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7hwcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 22 19:05:45.502723 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:45.502548 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" podUID="3179ecc9-7c67-46ac-bb01-09336685a9a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 22 19:05:46.502841 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:46.502360 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7hwcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 22 19:05:46.502841 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:46.502439 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" podUID="3179ecc9-7c67-46ac-bb01-09336685a9a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 22 19:05:47.502407 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:47.502375 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7hwcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 22 19:05:47.502599 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:47.502432 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" podUID="3179ecc9-7c67-46ac-bb01-09336685a9a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 22 19:05:48.506797 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:48.506763 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:05:48.507257 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:48.507109 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:05:48.507983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:05:48.507967 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7hwcv" Jul 22 19:06:09.169611 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:09.169585 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:06:09.171098 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:09.171074 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:06:19.545843 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.545781 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd"] Jul 22 19:06:19.549302 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.549279 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.552754 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.552718 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 19:06:19.552874 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.552757 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kcxtp\"" Jul 22 19:06:19.552874 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.552779 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 22 19:06:19.552874 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.552759 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 19:06:19.558083 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.558059 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd"] Jul 22 19:06:19.716151 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.716114 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.716151 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.716154 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.716384 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.716188 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.716384 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.716244 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.716384 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.716273 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.716384 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.716291 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsz7f\" (UniqueName: \"kubernetes.io/projected/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-kube-api-access-rsz7f\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.816916 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.816833 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.816916 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.816882 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.817127 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.816992 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.817127 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.817055 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.817127 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.817096 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsz7f\" (UniqueName: \"kubernetes.io/projected/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-kube-api-access-rsz7f\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.817265 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.817170 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.817265 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.817208 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.817384 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.817368 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.817490 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.817471 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.819365 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.819342 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.819684 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.819666 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.828626 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.828602 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsz7f\" (UniqueName: \"kubernetes.io/projected/d132cf56-cbc5-4d55-b04f-4e90a7b0194e-kube-api-access-rsz7f\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wbqtd\" (UID: \"d132cf56-cbc5-4d55-b04f-4e90a7b0194e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.859635 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.859610 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:19.994773 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.994744 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd"] Jul 22 19:06:19.996298 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:06:19.996268 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd132cf56_cbc5_4d55_b04f_4e90a7b0194e.slice/crio-c0633cef74698c70aa3bf017b37eb35433ea04ea5685482db491a3faadd04d75 WatchSource:0}: Error finding container c0633cef74698c70aa3bf017b37eb35433ea04ea5685482db491a3faadd04d75: Status 404 returned error can't find the container with id c0633cef74698c70aa3bf017b37eb35433ea04ea5685482db491a3faadd04d75 Jul 22 19:06:19.998139 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:19.998090 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 19:06:20.509746 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:20.509701 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" event={"ID":"d132cf56-cbc5-4d55-b04f-4e90a7b0194e","Type":"ContainerStarted","Data":"c0633cef74698c70aa3bf017b37eb35433ea04ea5685482db491a3faadd04d75"} Jul 22 19:06:21.516634 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:21.516545 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" event={"ID":"d132cf56-cbc5-4d55-b04f-4e90a7b0194e","Type":"ContainerStarted","Data":"e9e004a908dd19a599023389af99dfda9ab1bc986675b47d2d4cc3122b5bcfe7"} Jul 22 19:06:21.534329 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:21.534279 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" podStartSLOduration=1.4218378870000001 podStartE2EDuration="2.534263769s" totalImagesPullingTime="1.112425882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:06:21.532727191 +0000 UTC m=+915.195301532" watchObservedRunningTime="2026-07-22 19:06:21.534263769 +0000 UTC m=+915.196838107" Jul 22 19:06:29.860377 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:29.860346 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:29.860850 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:29.860405 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:29.873422 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:29.873397 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:06:30.571099 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:06:30.571070 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wbqtd" Jul 22 19:10:03.258828 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.258787 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8c7c85cd9-588pz"] Jul 22 19:10:03.262200 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.262181 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:03.264955 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.264930 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 19:10:03.266087 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.266069 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mmqq9\"" Jul 22 19:10:03.266203 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.266105 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 19:10:03.272649 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.272624 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8c7c85cd9-588pz"] Jul 22 19:10:03.339576 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.339500 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-webhook-cert\") pod \"maas-controller-8c7c85cd9-588pz\" (UID: \"ac02e249-b6c0-42c7-a708-8f2c9e698ca2\") " pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:03.339771 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.339678 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-422dp\" (UniqueName: \"kubernetes.io/projected/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-kube-api-access-422dp\") pod \"maas-controller-8c7c85cd9-588pz\" (UID: \"ac02e249-b6c0-42c7-a708-8f2c9e698ca2\") " pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:03.440352 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.440311 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-webhook-cert\") pod \"maas-controller-8c7c85cd9-588pz\" (UID: \"ac02e249-b6c0-42c7-a708-8f2c9e698ca2\") " pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:03.440577 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.440396 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-422dp\" (UniqueName: \"kubernetes.io/projected/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-kube-api-access-422dp\") pod \"maas-controller-8c7c85cd9-588pz\" (UID: \"ac02e249-b6c0-42c7-a708-8f2c9e698ca2\") " pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:03.442942 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.442919 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-webhook-cert\") pod \"maas-controller-8c7c85cd9-588pz\" (UID: \"ac02e249-b6c0-42c7-a708-8f2c9e698ca2\") " pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:03.448952 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.448926 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-422dp\" (UniqueName: \"kubernetes.io/projected/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-kube-api-access-422dp\") pod \"maas-controller-8c7c85cd9-588pz\" (UID: \"ac02e249-b6c0-42c7-a708-8f2c9e698ca2\") " pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:03.573311 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.573215 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:03.713644 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:03.713618 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8c7c85cd9-588pz"] Jul 22 19:10:03.715740 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:10:03.715706 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac02e249_b6c0_42c7_a708_8f2c9e698ca2.slice/crio-b2b15ee0a874ea3d1e375e3a4481828807160200fc76fd9fb0bf5f4d89dc029a WatchSource:0}: Error finding container b2b15ee0a874ea3d1e375e3a4481828807160200fc76fd9fb0bf5f4d89dc029a: Status 404 returned error can't find the container with id b2b15ee0a874ea3d1e375e3a4481828807160200fc76fd9fb0bf5f4d89dc029a Jul 22 19:10:04.628882 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:04.628845 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8c7c85cd9-588pz" event={"ID":"ac02e249-b6c0-42c7-a708-8f2c9e698ca2","Type":"ContainerStarted","Data":"b2b15ee0a874ea3d1e375e3a4481828807160200fc76fd9fb0bf5f4d89dc029a"} Jul 22 19:10:06.640345 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:06.640300 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8c7c85cd9-588pz" event={"ID":"ac02e249-b6c0-42c7-a708-8f2c9e698ca2","Type":"ContainerStarted","Data":"8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f"} Jul 22 19:10:06.640872 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:06.640413 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:06.660675 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:06.660618 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8c7c85cd9-588pz" podStartSLOduration=1.691805043 podStartE2EDuration="3.660601496s" totalImagesPullingTime="1.968796453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:10:06.658909652 +0000 UTC m=+1140.321484014" watchObservedRunningTime="2026-07-22 19:10:06.660601496 +0000 UTC m=+1140.323175834" Jul 22 19:10:17.648737 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:17.648704 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:22.300293 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.300248 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8c7c85cd9-588pz"] Jul 22 19:10:22.300717 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.300449 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8c7c85cd9-588pz" podUID="ac02e249-b6c0-42c7-a708-8f2c9e698ca2" containerName="manager" containerID="cri-o://8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f" gracePeriod=30 Jul 22 19:10:22.563267 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.563211 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:22.617950 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.617917 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-webhook-cert\") pod \"ac02e249-b6c0-42c7-a708-8f2c9e698ca2\" (UID: \"ac02e249-b6c0-42c7-a708-8f2c9e698ca2\") " Jul 22 19:10:22.618127 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.617972 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-422dp\" (UniqueName: \"kubernetes.io/projected/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-kube-api-access-422dp\") pod \"ac02e249-b6c0-42c7-a708-8f2c9e698ca2\" (UID: \"ac02e249-b6c0-42c7-a708-8f2c9e698ca2\") " Jul 22 19:10:22.620208 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.620172 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-kube-api-access-422dp" (OuterVolumeSpecName: "kube-api-access-422dp") pod "ac02e249-b6c0-42c7-a708-8f2c9e698ca2" (UID: "ac02e249-b6c0-42c7-a708-8f2c9e698ca2"). InnerVolumeSpecName "kube-api-access-422dp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:10:22.620321 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.620203 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ac02e249-b6c0-42c7-a708-8f2c9e698ca2" (UID: "ac02e249-b6c0-42c7-a708-8f2c9e698ca2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:10:22.719054 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.719021 2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-webhook-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:10:22.719054 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.719049 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-422dp\" (UniqueName: \"kubernetes.io/projected/ac02e249-b6c0-42c7-a708-8f2c9e698ca2-kube-api-access-422dp\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:10:22.722268 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.722237 2584 generic.go:356] "Generic (PLEG): container finished" podID="ac02e249-b6c0-42c7-a708-8f2c9e698ca2" containerID="8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f" exitCode=0 Jul 22 19:10:22.722383 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.722299 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8c7c85cd9-588pz" Jul 22 19:10:22.722383 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.722324 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8c7c85cd9-588pz" event={"ID":"ac02e249-b6c0-42c7-a708-8f2c9e698ca2","Type":"ContainerDied","Data":"8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f"} Jul 22 19:10:22.722455 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.722383 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8c7c85cd9-588pz" event={"ID":"ac02e249-b6c0-42c7-a708-8f2c9e698ca2","Type":"ContainerDied","Data":"b2b15ee0a874ea3d1e375e3a4481828807160200fc76fd9fb0bf5f4d89dc029a"} Jul 22 19:10:22.722455 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.722407 2584 scope.go:117] "RemoveContainer" containerID="8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f" Jul 22 19:10:22.731468 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.731451 2584 scope.go:117] "RemoveContainer" containerID="8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f" Jul 22 19:10:22.731766 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:10:22.731748 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f\": container with ID starting with 8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f not found: ID does not exist" containerID="8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f" Jul 22 19:10:22.731839 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.731774 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f"} err="failed to get container status \"8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f\": rpc error: code = NotFound desc = could not find container \"8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f\": container with ID starting with 8bb850665a9f238273391fbbc6cd3816927f45be2c27693367cf1f39f1519b8f not found: ID does not exist" Jul 22 19:10:22.744082 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.744057 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8c7c85cd9-588pz"] Jul 22 19:10:22.749574 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.749553 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8c7c85cd9-588pz"] Jul 22 19:10:22.906301 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:22.906267 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac02e249-b6c0-42c7-a708-8f2c9e698ca2" path="/var/lib/kubelet/pods/ac02e249-b6c0-42c7-a708-8f2c9e698ca2/volumes" Jul 22 19:10:23.506149 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.506111 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8c7c85cd9-6plrk"] Jul 22 19:10:23.506617 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.506434 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac02e249-b6c0-42c7-a708-8f2c9e698ca2" containerName="manager" Jul 22 19:10:23.506617 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.506444 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac02e249-b6c0-42c7-a708-8f2c9e698ca2" containerName="manager" Jul 22 19:10:23.506617 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.506511 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac02e249-b6c0-42c7-a708-8f2c9e698ca2" containerName="manager" Jul 22 19:10:23.511495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.511477 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8c7c85cd9-6plrk" Jul 22 19:10:23.515341 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.515316 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 19:10:23.515498 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.515356 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 19:10:23.515498 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.515355 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mmqq9\"" Jul 22 19:10:23.519424 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.519190 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8c7c85cd9-6plrk"] Jul 22 19:10:23.627219 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.627182 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e940eddf-66f1-4ade-ac0a-f973e3b0c906-webhook-cert\") pod \"maas-controller-8c7c85cd9-6plrk\" (UID: \"e940eddf-66f1-4ade-ac0a-f973e3b0c906\") " pod="opendatahub/maas-controller-8c7c85cd9-6plrk" Jul 22 19:10:23.627380 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.627244 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgmzg\" (UniqueName: \"kubernetes.io/projected/e940eddf-66f1-4ade-ac0a-f973e3b0c906-kube-api-access-wgmzg\") pod \"maas-controller-8c7c85cd9-6plrk\" (UID: \"e940eddf-66f1-4ade-ac0a-f973e3b0c906\") " pod="opendatahub/maas-controller-8c7c85cd9-6plrk" Jul 22 19:10:23.730652 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.730015 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgmzg\" (UniqueName: \"kubernetes.io/projected/e940eddf-66f1-4ade-ac0a-f973e3b0c906-kube-api-access-wgmzg\") pod \"maas-controller-8c7c85cd9-6plrk\" (UID: \"e940eddf-66f1-4ade-ac0a-f973e3b0c906\") " pod="opendatahub/maas-controller-8c7c85cd9-6plrk" Jul 22 19:10:23.730652 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.730178 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e940eddf-66f1-4ade-ac0a-f973e3b0c906-webhook-cert\") pod \"maas-controller-8c7c85cd9-6plrk\" (UID: \"e940eddf-66f1-4ade-ac0a-f973e3b0c906\") " pod="opendatahub/maas-controller-8c7c85cd9-6plrk" Jul 22 19:10:23.733587 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.733559 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e940eddf-66f1-4ade-ac0a-f973e3b0c906-webhook-cert\") pod \"maas-controller-8c7c85cd9-6plrk\" (UID: \"e940eddf-66f1-4ade-ac0a-f973e3b0c906\") " pod="opendatahub/maas-controller-8c7c85cd9-6plrk" Jul 22 19:10:23.739438 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.739413 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgmzg\" (UniqueName: \"kubernetes.io/projected/e940eddf-66f1-4ade-ac0a-f973e3b0c906-kube-api-access-wgmzg\") pod \"maas-controller-8c7c85cd9-6plrk\" (UID: \"e940eddf-66f1-4ade-ac0a-f973e3b0c906\") " pod="opendatahub/maas-controller-8c7c85cd9-6plrk" Jul 22 19:10:23.823641 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.823551 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8c7c85cd9-6plrk" Jul 22 19:10:23.965924 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:23.965898 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8c7c85cd9-6plrk"] Jul 22 19:10:23.967705 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:10:23.967678 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode940eddf_66f1_4ade_ac0a_f973e3b0c906.slice/crio-7efbf7e8973bdfc0c31f22c7c60911e10d7eb15b05739f45bcf718915a34c7c0 WatchSource:0}: Error finding container 7efbf7e8973bdfc0c31f22c7c60911e10d7eb15b05739f45bcf718915a34c7c0: Status 404 returned error can't find the container with id 7efbf7e8973bdfc0c31f22c7c60911e10d7eb15b05739f45bcf718915a34c7c0 Jul 22 19:10:24.736625 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:24.736584 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8c7c85cd9-6plrk" event={"ID":"e940eddf-66f1-4ade-ac0a-f973e3b0c906","Type":"ContainerStarted","Data":"3c661f34ac0a90f5f04abc70d5ff8817604b6f561aa7b5304c7f6d7e634b3af4"} Jul 22 19:10:24.737006 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:24.736646 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8c7c85cd9-6plrk" event={"ID":"e940eddf-66f1-4ade-ac0a-f973e3b0c906","Type":"ContainerStarted","Data":"7efbf7e8973bdfc0c31f22c7c60911e10d7eb15b05739f45bcf718915a34c7c0"} Jul 22 19:10:24.737006 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:24.736791 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8c7c85cd9-6plrk" Jul 22 19:10:24.755835 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:24.755787 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8c7c85cd9-6plrk" podStartSLOduration=1.1611329750000001 podStartE2EDuration="1.755767287s" totalImagesPullingTime="594.634312ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:10:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:10:24.753946644 +0000 UTC m=+1158.416520982" watchObservedRunningTime="2026-07-22 19:10:24.755767287 +0000 UTC m=+1158.418341626" Jul 22 19:10:35.744331 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:10:35.744299 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8c7c85cd9-6plrk" Jul 22 19:11:09.202179 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:11:09.202150 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:11:09.205313 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:11:09.205291 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:16:09.235291 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:16:09.235262 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:16:09.239048 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:16:09.239027 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:21:09.267589 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:21:09.267460 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:21:09.274397 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:21:09.271959 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:22:54.536418 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.536381 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd"] Jul 22 19:22:54.539956 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.539933 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.543087 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.543057 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3b5a0ccd-openshift-default-dockercfg-7lrc6\"" Jul 22 19:22:54.560246 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.560223 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd"] Jul 22 19:22:54.655491 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.655453 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-socket\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.655672 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.655505 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-data\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.655672 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.655540 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be50c419-a458-4752-ba75-177ba1b4bffc-istio-podinfo\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.655672 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.655602 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-certs\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.655672 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.655625 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkxph\" (UniqueName: \"kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-kube-api-access-nkxph\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.655837 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.655722 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-envoy\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.655837 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.655767 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be50c419-a458-4752-ba75-177ba1b4bffc-istiod-ca-cert\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.655837 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.655810 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-istio-token\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.655953 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.655859 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-credential-socket\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.756739 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.756704 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-socket\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.756913 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.756760 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-data\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.756913 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.756782 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be50c419-a458-4752-ba75-177ba1b4bffc-istio-podinfo\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.756913 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.756805 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-certs\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.756913 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.756826 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkxph\" (UniqueName: \"kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-kube-api-access-nkxph\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.756913 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.756879 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-envoy\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.756913 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.756909 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be50c419-a458-4752-ba75-177ba1b4bffc-istiod-ca-cert\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.757214 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.756941 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-istio-token\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.757214 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.756985 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-credential-socket\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.757214 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.757147 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-socket\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.757214 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.757195 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-certs\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.757542 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.757294 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-credential-socket\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.757542 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.757490 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-data\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.757801 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.757757 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be50c419-a458-4752-ba75-177ba1b4bffc-istiod-ca-cert\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.759476 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.759451 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-envoy\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.759614 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.759597 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be50c419-a458-4752-ba75-177ba1b4bffc-istio-podinfo\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.764999 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.764972 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-istio-token\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.765275 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.765257 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkxph\" (UniqueName: \"kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-kube-api-access-nkxph\") pod \"e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.850412 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.850380 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:54.989186 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.989160 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd"] Jul 22 19:22:54.990470 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:22:54.990437 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe50c419_a458_4752_ba75_177ba1b4bffc.slice/crio-3d9f868e6a060f4797e25bb58e13bff4c9f8b4ff2d1ee703771d2ab3ce5cea72 WatchSource:0}: Error finding container 3d9f868e6a060f4797e25bb58e13bff4c9f8b4ff2d1ee703771d2ab3ce5cea72: Status 404 returned error can't find the container with id 3d9f868e6a060f4797e25bb58e13bff4c9f8b4ff2d1ee703771d2ab3ce5cea72 Jul 22 19:22:54.992723 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.992686 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:22:54.992807 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.992768 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:22:54.992847 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:54.992809 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:22:55.569822 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:55.569783 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" event={"ID":"be50c419-a458-4752-ba75-177ba1b4bffc","Type":"ContainerStarted","Data":"d19dd592e07f0419c05433bb30e3718d427806da1a1cbc848876436980c90c28"} Jul 22 19:22:55.570225 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:55.569846 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" event={"ID":"be50c419-a458-4752-ba75-177ba1b4bffc","Type":"ContainerStarted","Data":"3d9f868e6a060f4797e25bb58e13bff4c9f8b4ff2d1ee703771d2ab3ce5cea72"} Jul 22 19:22:55.592666 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:55.592625 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" podStartSLOduration=1.592608841 podStartE2EDuration="1.592608841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:22:55.591847226 +0000 UTC m=+1909.254421594" watchObservedRunningTime="2026-07-22 19:22:55.592608841 +0000 UTC m=+1909.255183243" Jul 22 19:22:55.851073 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:55.851044 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:56.855870 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:56.855842 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:57.146639 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.146561 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82"] Jul 22 19:22:57.150528 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.150496 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:57.153109 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.153083 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-3b5a0ccd-serving-cert\"" Jul 22 19:22:57.153389 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.153202 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-65h52\"" Jul 22 19:22:57.160759 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.160723 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82"] Jul 22 19:22:57.178792 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.178760 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f153875c-79db-4cba-be41-aa3ee431abc3-openshift-service-ca\") pod \"maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:57.178950 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.178801 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f153875c-79db-4cba-be41-aa3ee431abc3-maas-api-tls\") pod \"maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:57.178950 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.178891 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psbqz\" (UniqueName: \"kubernetes.io/projected/f153875c-79db-4cba-be41-aa3ee431abc3-kube-api-access-psbqz\") pod \"maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:57.279450 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.279408 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f153875c-79db-4cba-be41-aa3ee431abc3-openshift-service-ca\") pod \"maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:57.279670 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.279459 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f153875c-79db-4cba-be41-aa3ee431abc3-maas-api-tls\") pod \"maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:57.279670 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.279532 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psbqz\" (UniqueName: \"kubernetes.io/projected/f153875c-79db-4cba-be41-aa3ee431abc3-kube-api-access-psbqz\") pod \"maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:57.279670 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:22:57.279643 2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-serving-cert: secret "maas-api-e2e-ait-3b5a0ccd-serving-cert" not found Jul 22 19:22:57.279867 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:22:57.279717 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f153875c-79db-4cba-be41-aa3ee431abc3-maas-api-tls podName:f153875c-79db-4cba-be41-aa3ee431abc3 nodeName:}" failed. No retries permitted until 2026-07-22 19:22:57.779694031 +0000 UTC m=+1911.442268349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f153875c-79db-4cba-be41-aa3ee431abc3-maas-api-tls") pod "maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" (UID: "f153875c-79db-4cba-be41-aa3ee431abc3") : secret "maas-api-e2e-ait-3b5a0ccd-serving-cert" not found Jul 22 19:22:57.280407 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.280369 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f153875c-79db-4cba-be41-aa3ee431abc3-openshift-service-ca\") pod \"maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:57.288659 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.288634 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psbqz\" (UniqueName: \"kubernetes.io/projected/f153875c-79db-4cba-be41-aa3ee431abc3-kube-api-access-psbqz\") pod \"maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:57.580436 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.580408 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:57.581507 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.581481 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:22:57.783499 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.783461 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f153875c-79db-4cba-be41-aa3ee431abc3-maas-api-tls\") pod \"maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:57.786273 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:57.786243 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f153875c-79db-4cba-be41-aa3ee431abc3-maas-api-tls\") pod \"maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:58.065450 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:58.065357 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:22:58.203019 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:58.202988 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82"] Jul 22 19:22:58.204822 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:22:58.204795 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf153875c_79db_4cba_be41_aa3ee431abc3.slice/crio-79bc6f94783144d35f7ee636ccb86417c3f31e5e5ceb70840044319c4782e575 WatchSource:0}: Error finding container 79bc6f94783144d35f7ee636ccb86417c3f31e5e5ceb70840044319c4782e575: Status 404 returned error can't find the container with id 79bc6f94783144d35f7ee636ccb86417c3f31e5e5ceb70840044319c4782e575 Jul 22 19:22:58.206118 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:58.206098 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 19:22:58.585825 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:58.585789 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" event={"ID":"f153875c-79db-4cba-be41-aa3ee431abc3","Type":"ContainerStarted","Data":"79bc6f94783144d35f7ee636ccb86417c3f31e5e5ceb70840044319c4782e575"} Jul 22 19:22:59.592392 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:59.592354 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" event={"ID":"f153875c-79db-4cba-be41-aa3ee431abc3","Type":"ContainerStarted","Data":"bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411"} Jul 22 19:22:59.610288 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:22:59.610241 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" podStartSLOduration=2.111437567 podStartE2EDuration="2.61022575s" totalImagesPullingTime="498.788183ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:22:59.608112505 +0000 UTC m=+1913.270686845" watchObservedRunningTime="2026-07-22 19:22:59.61022575 +0000 UTC m=+1913.272800088" Jul 22 19:23:00.597762 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:00.597719 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:23:06.610772 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:06.610737 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:23:12.582287 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:12.582251 2584 patch_prober.go:28] interesting pod/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 22 19:23:12.582705 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:12.582306 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 22 19:23:12.666967 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:12.666932 2584 generic.go:356] "Generic (PLEG): container finished" podID="be50c419-a458-4752-ba75-177ba1b4bffc" containerID="d19dd592e07f0419c05433bb30e3718d427806da1a1cbc848876436980c90c28" exitCode=0 Jul 22 19:23:12.667118 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:12.667007 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" event={"ID":"be50c419-a458-4752-ba75-177ba1b4bffc","Type":"ContainerDied","Data":"d19dd592e07f0419c05433bb30e3718d427806da1a1cbc848876436980c90c28"} Jul 22 19:23:12.667403 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:12.667388 2584 scope.go:117] "RemoveContainer" containerID="d19dd592e07f0419c05433bb30e3718d427806da1a1cbc848876436980c90c28" Jul 22 19:23:12.668386 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:12.668359 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:23:12.668442 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:12.668430 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:23:12.668486 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:12.668474 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:23:13.674559 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:13.674501 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" event={"ID":"be50c419-a458-4752-ba75-177ba1b4bffc","Type":"ContainerStarted","Data":"52bb2ca920bb12b82ec9be6562d66035e3a6091d0b51fcaa996aed87dafdd7f8"} Jul 22 19:23:13.850591 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:13.850554 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:23:13.850785 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:13.850763 2584 patch_prober.go:28] interesting pod/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 22 19:23:13.850834 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:13.850810 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 22 19:23:14.851335 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:14.851295 2584 patch_prober.go:28] interesting pod/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 22 19:23:14.851759 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:14.851358 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 22 19:23:15.851264 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:15.851226 2584 patch_prober.go:28] interesting pod/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 22 19:23:15.851468 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:15.851295 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 22 19:23:16.850938 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:16.850900 2584 patch_prober.go:28] interesting pod/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 22 19:23:16.851133 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:16.850956 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 22 19:23:17.855424 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:17.855395 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:23:17.855850 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:17.855739 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:23:17.856475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:17.856457 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:23:32.854820 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:32.854788 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82"] Jul 22 19:23:32.855356 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:32.855010 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" podUID="f153875c-79db-4cba-be41-aa3ee431abc3" containerName="maas-api" containerID="cri-o://bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411" gracePeriod=30 Jul 22 19:23:33.110015 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.109958 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:23:33.193944 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.193909 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f153875c-79db-4cba-be41-aa3ee431abc3-maas-api-tls\") pod \"f153875c-79db-4cba-be41-aa3ee431abc3\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " Jul 22 19:23:33.194109 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.194021 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f153875c-79db-4cba-be41-aa3ee431abc3-openshift-service-ca\") pod \"f153875c-79db-4cba-be41-aa3ee431abc3\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " Jul 22 19:23:33.194109 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.194054 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psbqz\" (UniqueName: \"kubernetes.io/projected/f153875c-79db-4cba-be41-aa3ee431abc3-kube-api-access-psbqz\") pod \"f153875c-79db-4cba-be41-aa3ee431abc3\" (UID: \"f153875c-79db-4cba-be41-aa3ee431abc3\") " Jul 22 19:23:33.194422 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.194395 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f153875c-79db-4cba-be41-aa3ee431abc3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f153875c-79db-4cba-be41-aa3ee431abc3" (UID: "f153875c-79db-4cba-be41-aa3ee431abc3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:23:33.196361 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.196329 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f153875c-79db-4cba-be41-aa3ee431abc3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f153875c-79db-4cba-be41-aa3ee431abc3" (UID: "f153875c-79db-4cba-be41-aa3ee431abc3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:23:33.196455 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.196396 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f153875c-79db-4cba-be41-aa3ee431abc3-kube-api-access-psbqz" (OuterVolumeSpecName: "kube-api-access-psbqz") pod "f153875c-79db-4cba-be41-aa3ee431abc3" (UID: "f153875c-79db-4cba-be41-aa3ee431abc3"). InnerVolumeSpecName "kube-api-access-psbqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:23:33.295011 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.294977 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f153875c-79db-4cba-be41-aa3ee431abc3-openshift-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:33.295011 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.295006 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psbqz\" (UniqueName: \"kubernetes.io/projected/f153875c-79db-4cba-be41-aa3ee431abc3-kube-api-access-psbqz\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:33.295011 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.295017 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f153875c-79db-4cba-be41-aa3ee431abc3-maas-api-tls\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:33.784259 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.784221 2584 generic.go:356] "Generic (PLEG): container finished" podID="f153875c-79db-4cba-be41-aa3ee431abc3" containerID="bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411" exitCode=0 Jul 22 19:23:33.784457 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.784274 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" Jul 22 19:23:33.784457 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.784278 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" event={"ID":"f153875c-79db-4cba-be41-aa3ee431abc3","Type":"ContainerDied","Data":"bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411"} Jul 22 19:23:33.784457 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.784411 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82" event={"ID":"f153875c-79db-4cba-be41-aa3ee431abc3","Type":"ContainerDied","Data":"79bc6f94783144d35f7ee636ccb86417c3f31e5e5ceb70840044319c4782e575"} Jul 22 19:23:33.784457 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.784447 2584 scope.go:117] "RemoveContainer" containerID="bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411" Jul 22 19:23:33.794778 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.794755 2584 scope.go:117] "RemoveContainer" containerID="bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411" Jul 22 19:23:33.795067 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:23:33.795048 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411\": container with ID starting with bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411 not found: ID does not exist" containerID="bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411" Jul 22 19:23:33.795112 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.795078 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411"} err="failed to get container status \"bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411\": rpc error: code = NotFound desc = could not find container \"bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411\": container with ID starting with bf2d613308557f17f70a2ec85f0a21a1d4f786407de4f24b6f7c09a2b05b6411 not found: ID does not exist" Jul 22 19:23:33.806240 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.806213 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82"] Jul 22 19:23:33.809890 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:33.809867 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-3b5a0ccd-79bc99d498-qpq82"] Jul 22 19:23:34.906445 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:34.906409 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f153875c-79db-4cba-be41-aa3ee431abc3" path="/var/lib/kubelet/pods/f153875c-79db-4cba-be41-aa3ee431abc3/volumes" Jul 22 19:23:37.698658 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:37.698622 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd"] Jul 22 19:23:37.699176 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:37.698825 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" containerID="cri-o://52bb2ca920bb12b82ec9be6562d66035e3a6091d0b51fcaa996aed87dafdd7f8" gracePeriod=30 Jul 22 19:23:38.686992 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.686960 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd"] Jul 22 19:23:38.687310 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.687298 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f153875c-79db-4cba-be41-aa3ee431abc3" containerName="maas-api" Jul 22 19:23:38.687361 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.687312 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f153875c-79db-4cba-be41-aa3ee431abc3" containerName="maas-api" Jul 22 19:23:38.687398 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.687381 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f153875c-79db-4cba-be41-aa3ee431abc3" containerName="maas-api" Jul 22 19:23:38.690324 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.690307 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.692685 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.692663 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-e886b356-gw-openshift-default-dockercfg-gb8st\"" Jul 22 19:23:38.701602 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.701478 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd"] Jul 22 19:23:38.849336 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.849306 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-socket\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.849336 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.849339 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-certs\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.849592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.849356 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-podinfo\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.849592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.849422 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-data\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.849592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.849461 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4n4c\" (UniqueName: \"kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-kube-api-access-g4n4c\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.849592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.849482 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-credential-socket\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.849592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.849502 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istiod-ca-cert\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.849592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.849546 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-envoy\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.849817 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.849612 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-token\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.951043 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.950943 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-data\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.951043 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.951010 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4n4c\" (UniqueName: \"kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-kube-api-access-g4n4c\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.951043 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.951043 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-credential-socket\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.951378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.951071 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istiod-ca-cert\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.951378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.951097 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-envoy\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.951378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.951161 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-token\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.951378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.951195 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-socket\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.951378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.951217 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-certs\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.951378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.951242 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-podinfo\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.952272 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.952245 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-credential-socket\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.952423 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.952397 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istiod-ca-cert\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.952606 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.952386 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-socket\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.952716 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.952546 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-data\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.952796 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.952708 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-certs\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.954744 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.954721 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-envoy\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.954865 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.954848 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-podinfo\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.959674 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.959648 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4n4c\" (UniqueName: \"kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-kube-api-access-g4n4c\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:38.960060 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:38.960039 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-token\") pod \"e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:39.002670 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:39.002648 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:39.141872 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:39.141845 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd"] Jul 22 19:23:39.143171 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:23:39.143145 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod061405a7_f75d_48b0_9f1e_e8158f9bfc2e.slice/crio-02f35c040a239fa9487cdfa2779eec95575e62de06692b34fda8085ba7837401 WatchSource:0}: Error finding container 02f35c040a239fa9487cdfa2779eec95575e62de06692b34fda8085ba7837401: Status 404 returned error can't find the container with id 02f35c040a239fa9487cdfa2779eec95575e62de06692b34fda8085ba7837401 Jul 22 19:23:39.145192 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:39.145160 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:23:39.145256 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:39.145244 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:23:39.145298 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:39.145275 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:23:39.818850 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:39.818812 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" event={"ID":"061405a7-f75d-48b0-9f1e-e8158f9bfc2e","Type":"ContainerStarted","Data":"6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636"} Jul 22 19:23:39.819232 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:39.818870 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" event={"ID":"061405a7-f75d-48b0-9f1e-e8158f9bfc2e","Type":"ContainerStarted","Data":"02f35c040a239fa9487cdfa2779eec95575e62de06692b34fda8085ba7837401"} Jul 22 19:23:39.839575 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:39.839505 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" podStartSLOduration=1.8394899919999999 podStartE2EDuration="1.839489992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:23:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:23:39.837560369 +0000 UTC m=+1953.500134710" watchObservedRunningTime="2026-07-22 19:23:39.839489992 +0000 UTC m=+1953.502064398" Jul 22 19:23:40.003093 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:40.003050 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:40.008048 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:40.008022 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:40.823911 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:40.823875 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:40.824953 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:40.824933 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:41.450760 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.450722 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs"] Jul 22 19:23:41.454925 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.454896 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.457531 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.457493 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e886b356-serving-cert\"" Jul 22 19:23:41.458720 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.458549 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-65h52\"" Jul 22 19:23:41.463044 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.463025 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs"] Jul 22 19:23:41.575011 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.574968 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-openshift-service-ca\") pod \"maas-api-e2e-migrate-e886b356-678649756f-rtgbs\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.575191 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.575127 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdhtc\" (UniqueName: \"kubernetes.io/projected/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-kube-api-access-tdhtc\") pod \"maas-api-e2e-migrate-e886b356-678649756f-rtgbs\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.575191 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.575183 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-maas-api-tls\") pod \"maas-api-e2e-migrate-e886b356-678649756f-rtgbs\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.675937 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.675900 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-openshift-service-ca\") pod \"maas-api-e2e-migrate-e886b356-678649756f-rtgbs\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.676107 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.675980 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdhtc\" (UniqueName: \"kubernetes.io/projected/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-kube-api-access-tdhtc\") pod \"maas-api-e2e-migrate-e886b356-678649756f-rtgbs\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.676107 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.676007 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-maas-api-tls\") pod \"maas-api-e2e-migrate-e886b356-678649756f-rtgbs\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.676646 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.676616 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-openshift-service-ca\") pod \"maas-api-e2e-migrate-e886b356-678649756f-rtgbs\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.678683 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.678655 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-maas-api-tls\") pod \"maas-api-e2e-migrate-e886b356-678649756f-rtgbs\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.684097 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.684073 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdhtc\" (UniqueName: \"kubernetes.io/projected/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-kube-api-access-tdhtc\") pod \"maas-api-e2e-migrate-e886b356-678649756f-rtgbs\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.768634 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.768557 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:41.915135 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:41.915104 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs"] Jul 22 19:23:41.917273 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:23:41.917245 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9a091b_9563_4059_8ea1_fa8c6b0336d3.slice/crio-4fdeb4626f7c76d563417b368a5982ac857dd517f2d9e28d4bea144ad6db74a4 WatchSource:0}: Error finding container 4fdeb4626f7c76d563417b368a5982ac857dd517f2d9e28d4bea144ad6db74a4: Status 404 returned error can't find the container with id 4fdeb4626f7c76d563417b368a5982ac857dd517f2d9e28d4bea144ad6db74a4 Jul 22 19:23:42.838400 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:42.838368 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" event={"ID":"fb9a091b-9563-4059-8ea1-fa8c6b0336d3","Type":"ContainerStarted","Data":"288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991"} Jul 22 19:23:42.838574 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:42.838425 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" event={"ID":"fb9a091b-9563-4059-8ea1-fa8c6b0336d3","Type":"ContainerStarted","Data":"4fdeb4626f7c76d563417b368a5982ac857dd517f2d9e28d4bea144ad6db74a4"} Jul 22 19:23:42.838574 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:42.838452 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:42.840450 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:42.840419 2584 generic.go:356] "Generic (PLEG): container finished" podID="be50c419-a458-4752-ba75-177ba1b4bffc" containerID="52bb2ca920bb12b82ec9be6562d66035e3a6091d0b51fcaa996aed87dafdd7f8" exitCode=0 Jul 22 19:23:42.840588 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:42.840495 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" event={"ID":"be50c419-a458-4752-ba75-177ba1b4bffc","Type":"ContainerDied","Data":"52bb2ca920bb12b82ec9be6562d66035e3a6091d0b51fcaa996aed87dafdd7f8"} Jul 22 19:23:42.840588 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:42.840569 2584 scope.go:117] "RemoveContainer" containerID="d19dd592e07f0419c05433bb30e3718d427806da1a1cbc848876436980c90c28" Jul 22 19:23:42.857747 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:42.857687 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" podStartSLOduration=1.4234707850000001 podStartE2EDuration="1.857669612s" totalImagesPullingTime="434.198827ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:23:42.853662857 +0000 UTC m=+1956.516237195" watchObservedRunningTime="2026-07-22 19:23:42.857669612 +0000 UTC m=+1956.520243951" Jul 22 19:23:42.962448 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:42.962309 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:23:43.095755 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.095720 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-socket\") pod \"be50c419-a458-4752-ba75-177ba1b4bffc\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " Jul 22 19:23:43.095755 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.095762 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-credential-socket\") pod \"be50c419-a458-4752-ba75-177ba1b4bffc\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " Jul 22 19:23:43.095983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.095796 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be50c419-a458-4752-ba75-177ba1b4bffc-istio-podinfo\") pod \"be50c419-a458-4752-ba75-177ba1b4bffc\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " Jul 22 19:23:43.095983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.095818 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-certs\") pod \"be50c419-a458-4752-ba75-177ba1b4bffc\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " Jul 22 19:23:43.095983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.095841 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be50c419-a458-4752-ba75-177ba1b4bffc-istiod-ca-cert\") pod \"be50c419-a458-4752-ba75-177ba1b4bffc\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " Jul 22 19:23:43.095983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.095872 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-envoy\") pod \"be50c419-a458-4752-ba75-177ba1b4bffc\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " Jul 22 19:23:43.095983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.095908 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-data\") pod \"be50c419-a458-4752-ba75-177ba1b4bffc\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " Jul 22 19:23:43.095983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.095953 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-istio-token\") pod \"be50c419-a458-4752-ba75-177ba1b4bffc\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " Jul 22 19:23:43.096291 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.096000 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "be50c419-a458-4752-ba75-177ba1b4bffc" (UID: "be50c419-a458-4752-ba75-177ba1b4bffc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:23:43.096291 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.096010 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkxph\" (UniqueName: \"kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-kube-api-access-nkxph\") pod \"be50c419-a458-4752-ba75-177ba1b4bffc\" (UID: \"be50c419-a458-4752-ba75-177ba1b4bffc\") " Jul 22 19:23:43.096291 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.096073 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "be50c419-a458-4752-ba75-177ba1b4bffc" (UID: "be50c419-a458-4752-ba75-177ba1b4bffc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:23:43.096291 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.096224 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "be50c419-a458-4752-ba75-177ba1b4bffc" (UID: "be50c419-a458-4752-ba75-177ba1b4bffc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:23:43.096548 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.096372 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be50c419-a458-4752-ba75-177ba1b4bffc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "be50c419-a458-4752-ba75-177ba1b4bffc" (UID: "be50c419-a458-4752-ba75-177ba1b4bffc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:23:43.096548 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.096445 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:43.096548 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.096466 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:43.096548 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.096480 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:43.096993 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.096968 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-data" (OuterVolumeSpecName: "istio-data") pod "be50c419-a458-4752-ba75-177ba1b4bffc" (UID: "be50c419-a458-4752-ba75-177ba1b4bffc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:23:43.098558 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.098531 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-istio-token" (OuterVolumeSpecName: "istio-token") pod "be50c419-a458-4752-ba75-177ba1b4bffc" (UID: "be50c419-a458-4752-ba75-177ba1b4bffc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:23:43.098649 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.098561 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-kube-api-access-nkxph" (OuterVolumeSpecName: "kube-api-access-nkxph") pod "be50c419-a458-4752-ba75-177ba1b4bffc" (UID: "be50c419-a458-4752-ba75-177ba1b4bffc"). InnerVolumeSpecName "kube-api-access-nkxph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:23:43.098854 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.098836 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be50c419-a458-4752-ba75-177ba1b4bffc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "be50c419-a458-4752-ba75-177ba1b4bffc" (UID: "be50c419-a458-4752-ba75-177ba1b4bffc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:23:43.098909 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.098847 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "be50c419-a458-4752-ba75-177ba1b4bffc" (UID: "be50c419-a458-4752-ba75-177ba1b4bffc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:23:43.197596 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.197565 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be50c419-a458-4752-ba75-177ba1b4bffc-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:43.197596 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.197591 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be50c419-a458-4752-ba75-177ba1b4bffc-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:43.197596 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.197600 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:43.197792 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.197607 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be50c419-a458-4752-ba75-177ba1b4bffc-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:43.197792 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.197617 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:43.197792 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.197625 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkxph\" (UniqueName: \"kubernetes.io/projected/be50c419-a458-4752-ba75-177ba1b4bffc-kube-api-access-nkxph\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:23:43.847591 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.847559 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" Jul 22 19:23:43.847793 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.847585 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd" event={"ID":"be50c419-a458-4752-ba75-177ba1b4bffc","Type":"ContainerDied","Data":"3d9f868e6a060f4797e25bb58e13bff4c9f8b4ff2d1ee703771d2ab3ce5cea72"} Jul 22 19:23:43.847793 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.847680 2584 scope.go:117] "RemoveContainer" containerID="52bb2ca920bb12b82ec9be6562d66035e3a6091d0b51fcaa996aed87dafdd7f8" Jul 22 19:23:43.869595 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.869570 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd"] Jul 22 19:23:43.875210 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:43.875181 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3b5a0ccd-openshift-default-6d9d6bb957-ttxgd"] Jul 22 19:23:44.907129 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:44.907095 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" path="/var/lib/kubelet/pods/be50c419-a458-4752-ba75-177ba1b4bffc/volumes" Jul 22 19:23:48.853397 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:48.853365 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:23:55.826052 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:55.826018 2584 patch_prober.go:28] interesting pod/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 22 19:23:55.826462 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:55.826088 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 22 19:23:55.918221 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:55.918187 2584 generic.go:356] "Generic (PLEG): container finished" podID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerID="6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636" exitCode=0 Jul 22 19:23:55.918379 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:55.918259 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" event={"ID":"061405a7-f75d-48b0-9f1e-e8158f9bfc2e","Type":"ContainerDied","Data":"6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636"} Jul 22 19:23:55.918667 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:55.918650 2584 scope.go:117] "RemoveContainer" containerID="6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636" Jul 22 19:23:55.919661 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:55.919632 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:23:55.919769 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:55.919714 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:23:55.919769 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:55.919763 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:23:56.924577 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:56.924483 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" event={"ID":"061405a7-f75d-48b0-9f1e-e8158f9bfc2e","Type":"ContainerStarted","Data":"22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b"} Jul 22 19:23:57.003871 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:57.003771 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:23:57.004084 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:57.004051 2584 patch_prober.go:28] interesting pod/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 22 19:23:57.004152 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:57.004108 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 22 19:23:58.003989 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:58.003952 2584 patch_prober.go:28] interesting pod/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 22 19:23:58.004614 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:58.004580 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 22 19:23:59.003279 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:59.003241 2584 patch_prober.go:28] interesting pod/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 22 19:23:59.003590 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:23:59.003559 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 22 19:24:00.004011 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:00.003965 2584 patch_prober.go:28] interesting pod/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 22 19:24:00.004535 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:00.004039 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 22 19:24:01.015333 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:01.015297 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:24:01.019743 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:01.019720 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:24:01.020632 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:01.020607 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:24:16.597960 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:16.596445 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs"] Jul 22 19:24:16.597960 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:16.596713 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" podUID="fb9a091b-9563-4059-8ea1-fa8c6b0336d3" containerName="maas-api" containerID="cri-o://288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991" gracePeriod=30 Jul 22 19:24:16.849329 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:16.849271 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:24:16.996377 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:16.996345 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-openshift-service-ca\") pod \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " Jul 22 19:24:16.996592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:16.996389 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdhtc\" (UniqueName: \"kubernetes.io/projected/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-kube-api-access-tdhtc\") pod \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " Jul 22 19:24:16.996592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:16.996457 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-maas-api-tls\") pod \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\" (UID: \"fb9a091b-9563-4059-8ea1-fa8c6b0336d3\") " Jul 22 19:24:16.996790 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:16.996759 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb9a091b-9563-4059-8ea1-fa8c6b0336d3" (UID: "fb9a091b-9563-4059-8ea1-fa8c6b0336d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:24:16.998721 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:16.998698 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb9a091b-9563-4059-8ea1-fa8c6b0336d3" (UID: "fb9a091b-9563-4059-8ea1-fa8c6b0336d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 19:24:16.998816 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:16.998761 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-kube-api-access-tdhtc" (OuterVolumeSpecName: "kube-api-access-tdhtc") pod "fb9a091b-9563-4059-8ea1-fa8c6b0336d3" (UID: "fb9a091b-9563-4059-8ea1-fa8c6b0336d3"). InnerVolumeSpecName "kube-api-access-tdhtc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:24:17.029446 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.029417 2584 generic.go:356] "Generic (PLEG): container finished" podID="fb9a091b-9563-4059-8ea1-fa8c6b0336d3" containerID="288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991" exitCode=0 Jul 22 19:24:17.029562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.029482 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" Jul 22 19:24:17.029562 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.029510 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" event={"ID":"fb9a091b-9563-4059-8ea1-fa8c6b0336d3","Type":"ContainerDied","Data":"288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991"} Jul 22 19:24:17.029639 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.029580 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs" event={"ID":"fb9a091b-9563-4059-8ea1-fa8c6b0336d3","Type":"ContainerDied","Data":"4fdeb4626f7c76d563417b368a5982ac857dd517f2d9e28d4bea144ad6db74a4"} Jul 22 19:24:17.029639 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.029605 2584 scope.go:117] "RemoveContainer" containerID="288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991" Jul 22 19:24:17.038553 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.038509 2584 scope.go:117] "RemoveContainer" containerID="288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991" Jul 22 19:24:17.038792 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:24:17.038770 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991\": container with ID starting with 288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991 not found: ID does not exist" containerID="288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991" Jul 22 19:24:17.038862 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.038804 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991"} err="failed to get container status \"288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991\": rpc error: code = NotFound desc = could not find container \"288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991\": container with ID starting with 288f8d73eaedf7ae77fe43a717d65c4faae65ea85bd2ad7c3259cddd2d8b1991 not found: ID does not exist" Jul 22 19:24:17.052047 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.052023 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs"] Jul 22 19:24:17.058357 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.058338 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e886b356-678649756f-rtgbs"] Jul 22 19:24:17.097296 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.097276 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-maas-api-tls\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:17.097296 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.097297 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-openshift-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:17.097411 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:17.097307 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdhtc\" (UniqueName: \"kubernetes.io/projected/fb9a091b-9563-4059-8ea1-fa8c6b0336d3-kube-api-access-tdhtc\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:18.906057 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:18.906024 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb9a091b-9563-4059-8ea1-fa8c6b0336d3" path="/var/lib/kubelet/pods/fb9a091b-9563-4059-8ea1-fa8c6b0336d3/volumes" Jul 22 19:24:21.443848 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:21.443804 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd"] Jul 22 19:24:21.444397 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:21.444061 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" containerID="cri-o://22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b" gracePeriod=30 Jul 22 19:24:22.211997 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.211959 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw"] Jul 22 19:24:22.212497 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.212482 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb9a091b-9563-4059-8ea1-fa8c6b0336d3" containerName="maas-api" Jul 22 19:24:22.212592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.212501 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb9a091b-9563-4059-8ea1-fa8c6b0336d3" containerName="maas-api" Jul 22 19:24:22.212592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.212541 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" Jul 22 19:24:22.212592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.212550 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" Jul 22 19:24:22.212592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.212563 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" Jul 22 19:24:22.212592 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.212571 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" Jul 22 19:24:22.212751 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.212664 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" Jul 22 19:24:22.212751 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.212675 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="be50c419-a458-4752-ba75-177ba1b4bffc" containerName="istio-proxy" Jul 22 19:24:22.212751 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.212683 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb9a091b-9563-4059-8ea1-fa8c6b0336d3" containerName="maas-api" Jul 22 19:24:22.217312 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.217292 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.219850 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.219827 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3df5cf99-openshift-default-dockercfg-rl8kx\"" Jul 22 19:24:22.224976 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.224949 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw"] Jul 22 19:24:22.351079 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.351043 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istiod-ca-cert\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.351079 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.351087 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-certs\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.351293 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.351142 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxpm2\" (UniqueName: \"kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-kube-api-access-lxpm2\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.351293 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.351180 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-envoy\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.351293 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.351228 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-token\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.351406 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.351300 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-socket\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.351406 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.351329 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-data\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.351406 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.351352 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-credential-socket\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.351406 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.351367 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-podinfo\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452207 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452175 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-token\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452623 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452249 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-socket\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452623 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452276 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-data\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452623 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452303 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-credential-socket\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452623 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452325 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-podinfo\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452623 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452390 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istiod-ca-cert\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452623 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452422 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-certs\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452971 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452651 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxpm2\" (UniqueName: \"kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-kube-api-access-lxpm2\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452971 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452697 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-envoy\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452971 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452751 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-socket\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452971 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452767 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-credential-socket\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452971 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452796 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-data\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.452971 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.452921 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-certs\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.453279 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.453210 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istiod-ca-cert\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.454937 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.454914 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-envoy\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.455032 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.454946 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-podinfo\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.460402 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.460378 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxpm2\" (UniqueName: \"kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-kube-api-access-lxpm2\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.460854 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.460834 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-token\") pod \"e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.529779 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.529690 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:22.668637 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.668608 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw"] Jul 22 19:24:22.670178 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:24:22.670152 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbcbb26c_9d76_4994_bef4_cfdb64abca9a.slice/crio-5c33aea40e9e1f130f10d51cbeb3a48867bba4de333677b1b19c7d49fda4fe44 WatchSource:0}: Error finding container 5c33aea40e9e1f130f10d51cbeb3a48867bba4de333677b1b19c7d49fda4fe44: Status 404 returned error can't find the container with id 5c33aea40e9e1f130f10d51cbeb3a48867bba4de333677b1b19c7d49fda4fe44 Jul 22 19:24:22.672296 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.672266 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:24:22.672363 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.672326 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:24:22.672363 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:22.672352 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:24:23.061641 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:23.061600 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" event={"ID":"dbcbb26c-9d76-4994-bef4-cfdb64abca9a","Type":"ContainerStarted","Data":"4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a"} Jul 22 19:24:23.061823 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:23.061660 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" event={"ID":"dbcbb26c-9d76-4994-bef4-cfdb64abca9a","Type":"ContainerStarted","Data":"5c33aea40e9e1f130f10d51cbeb3a48867bba4de333677b1b19c7d49fda4fe44"} Jul 22 19:24:23.085002 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:23.084950 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" podStartSLOduration=1.084933076 podStartE2EDuration="1.084933076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:24:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:24:23.082773495 +0000 UTC m=+1996.745347835" watchObservedRunningTime="2026-07-22 19:24:23.084933076 +0000 UTC m=+1996.747507414" Jul 22 19:24:23.530239 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:23.530204 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:23.535221 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:23.535194 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:24.067092 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:24.067060 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:24.067882 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:24.067864 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:26.689922 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.689898 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:24:26.790787 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.790700 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-podinfo\") pod \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " Jul 22 19:24:26.790787 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.790743 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istiod-ca-cert\") pod \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " Jul 22 19:24:26.790787 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.790779 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-certs\") pod \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " Jul 22 19:24:26.791053 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.790849 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-socket\") pod \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " Jul 22 19:24:26.791053 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.790873 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-data\") pod \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " Jul 22 19:24:26.791053 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.790923 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4n4c\" (UniqueName: \"kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-kube-api-access-g4n4c\") pod \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " Jul 22 19:24:26.791053 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.790957 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-credential-socket\") pod \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " Jul 22 19:24:26.791053 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.791015 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-envoy\") pod \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " Jul 22 19:24:26.791053 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.791038 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-token\") pod \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\" (UID: \"061405a7-f75d-48b0-9f1e-e8158f9bfc2e\") " Jul 22 19:24:26.791354 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.791124 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "061405a7-f75d-48b0-9f1e-e8158f9bfc2e" (UID: "061405a7-f75d-48b0-9f1e-e8158f9bfc2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:26.791354 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.791133 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "061405a7-f75d-48b0-9f1e-e8158f9bfc2e" (UID: "061405a7-f75d-48b0-9f1e-e8158f9bfc2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:26.791354 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.791232 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "061405a7-f75d-48b0-9f1e-e8158f9bfc2e" (UID: "061405a7-f75d-48b0-9f1e-e8158f9bfc2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:24:26.791504 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.791352 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "061405a7-f75d-48b0-9f1e-e8158f9bfc2e" (UID: "061405a7-f75d-48b0-9f1e-e8158f9bfc2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:26.791504 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.791493 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:26.791650 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.791535 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:26.791650 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.791551 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:26.791650 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.791564 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:26.792249 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.792210 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-data" (OuterVolumeSpecName: "istio-data") pod "061405a7-f75d-48b0-9f1e-e8158f9bfc2e" (UID: "061405a7-f75d-48b0-9f1e-e8158f9bfc2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:26.793735 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.793703 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "061405a7-f75d-48b0-9f1e-e8158f9bfc2e" (UID: "061405a7-f75d-48b0-9f1e-e8158f9bfc2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:26.793862 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.793778 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "061405a7-f75d-48b0-9f1e-e8158f9bfc2e" (UID: "061405a7-f75d-48b0-9f1e-e8158f9bfc2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:24:26.793862 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.793796 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-token" (OuterVolumeSpecName: "istio-token") pod "061405a7-f75d-48b0-9f1e-e8158f9bfc2e" (UID: "061405a7-f75d-48b0-9f1e-e8158f9bfc2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:24:26.793955 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.793931 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-kube-api-access-g4n4c" (OuterVolumeSpecName: "kube-api-access-g4n4c") pod "061405a7-f75d-48b0-9f1e-e8158f9bfc2e" (UID: "061405a7-f75d-48b0-9f1e-e8158f9bfc2e"). InnerVolumeSpecName "kube-api-access-g4n4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:24:26.892387 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.892361 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:26.892387 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.892386 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:26.892598 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.892396 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:26.892598 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.892406 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:26.892598 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:26.892415 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4n4c\" (UniqueName: \"kubernetes.io/projected/061405a7-f75d-48b0-9f1e-e8158f9bfc2e-kube-api-access-g4n4c\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:24:27.084495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.084457 2584 generic.go:356] "Generic (PLEG): container finished" podID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerID="22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b" exitCode=0 Jul 22 19:24:27.084671 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.084506 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" Jul 22 19:24:27.084671 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.084550 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" event={"ID":"061405a7-f75d-48b0-9f1e-e8158f9bfc2e","Type":"ContainerDied","Data":"22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b"} Jul 22 19:24:27.084671 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.084607 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd" event={"ID":"061405a7-f75d-48b0-9f1e-e8158f9bfc2e","Type":"ContainerDied","Data":"02f35c040a239fa9487cdfa2779eec95575e62de06692b34fda8085ba7837401"} Jul 22 19:24:27.084671 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.084635 2584 scope.go:117] "RemoveContainer" containerID="22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b" Jul 22 19:24:27.094606 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.094585 2584 scope.go:117] "RemoveContainer" containerID="6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636" Jul 22 19:24:27.103267 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.103246 2584 scope.go:117] "RemoveContainer" containerID="22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b" Jul 22 19:24:27.103576 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:24:27.103553 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b\": container with ID starting with 22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b not found: ID does not exist" containerID="22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b" Jul 22 19:24:27.103671 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.103589 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b"} err="failed to get container status \"22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b\": rpc error: code = NotFound desc = could not find container \"22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b\": container with ID starting with 22b110b97f87c4efa19948c552cf9fb195af1e9afc259891af26808d31bd972b not found: ID does not exist" Jul 22 19:24:27.103671 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.103612 2584 scope.go:117] "RemoveContainer" containerID="6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636" Jul 22 19:24:27.103873 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:24:27.103856 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636\": container with ID starting with 6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636 not found: ID does not exist" containerID="6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636" Jul 22 19:24:27.103915 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.103880 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636"} err="failed to get container status \"6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636\": rpc error: code = NotFound desc = could not find container \"6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636\": container with ID starting with 6991e1fe2ce7ba73a4193a81dd121b10b9fd793962ead0bbd745d9e4487c7636 not found: ID does not exist" Jul 22 19:24:27.104092 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.104075 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd"] Jul 22 19:24:27.108218 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:27.108194 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-e886b356-gw-openshift-default-65866d9999-sfljd"] Jul 22 19:24:28.906467 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:28.906423 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" path="/var/lib/kubelet/pods/061405a7-f75d-48b0-9f1e-e8158f9bfc2e/volumes" Jul 22 19:24:39.900483 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:39.900450 2584 patch_prober.go:28] interesting pod/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": read tcp 10.134.0.2:48894->10.134.0.47:15021: read: connection reset by peer" start-of-body= Jul 22 19:24:39.901137 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:39.900542 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": read tcp 10.134.0.2:48894->10.134.0.47:15021: read: connection reset by peer" Jul 22 19:24:40.155563 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:40.155469 2584 generic.go:356] "Generic (PLEG): container finished" podID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerID="4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a" exitCode=0 Jul 22 19:24:40.155738 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:40.155549 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" event={"ID":"dbcbb26c-9d76-4994-bef4-cfdb64abca9a","Type":"ContainerDied","Data":"4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a"} Jul 22 19:24:40.155983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:40.155966 2584 scope.go:117] "RemoveContainer" containerID="4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a" Jul 22 19:24:40.157029 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:40.157001 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:24:40.157135 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:40.157081 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:24:40.157199 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:40.157136 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:24:41.162557 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:41.162492 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" event={"ID":"dbcbb26c-9d76-4994-bef4-cfdb64abca9a","Type":"ContainerStarted","Data":"116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74"} Jul 22 19:24:41.530229 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:41.530143 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:41.530404 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:41.530342 2584 patch_prober.go:28] interesting pod/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 22 19:24:41.530404 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:41.530391 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 22 19:24:42.530590 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:42.530550 2584 patch_prober.go:28] interesting pod/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 22 19:24:42.531002 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:42.530617 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 22 19:24:43.530318 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:43.530280 2584 patch_prober.go:28] interesting pod/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 22 19:24:43.530503 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:43.530337 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 22 19:24:44.530829 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:44.530789 2584 patch_prober.go:28] interesting pod/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 22 19:24:44.531216 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:44.530870 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 22 19:24:45.530912 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:45.530865 2584 patch_prober.go:28] interesting pod/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 22 19:24:45.531336 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:45.530943 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 22 19:24:46.533462 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:46.533426 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:46.533907 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:46.533720 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:24:46.534398 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:24:46.534381 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:25:06.687211 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:06.687170 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw"] Jul 22 19:25:06.687780 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:06.687459 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" containerID="cri-o://116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74" gracePeriod=30 Jul 22 19:25:07.585576 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.585540 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj"] Jul 22 19:25:07.585907 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.585894 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" Jul 22 19:25:07.585954 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.585909 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" Jul 22 19:25:07.585997 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.585974 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" Jul 22 19:25:07.585997 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.585984 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" Jul 22 19:25:07.588620 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.588599 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.592105 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.592001 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-bb81af95-openshift-default-dockercfg-ztzr9\"" Jul 22 19:25:07.601613 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.601588 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj"] Jul 22 19:25:07.744655 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.744613 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-data\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.745050 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.744692 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-envoy\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.745050 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.744725 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-token\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.745050 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.744769 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-socket\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.745050 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.744816 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-credential-socket\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.745050 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.744852 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6748e7d-aa1d-417c-bc7d-071cc5033646-istiod-ca-cert\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.745050 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.744884 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-podinfo\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.745050 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.744906 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrcsq\" (UniqueName: \"kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-kube-api-access-rrcsq\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.745050 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.744962 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-certs\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.845674 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.845583 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-envoy\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.845674 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.845624 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-token\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.845886 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.845772 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-socket\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.845886 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.845822 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-credential-socket\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.845886 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.845850 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6748e7d-aa1d-417c-bc7d-071cc5033646-istiod-ca-cert\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.846054 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.845890 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-podinfo\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.846054 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.846013 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrcsq\" (UniqueName: \"kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-kube-api-access-rrcsq\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.846168 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.846066 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-certs\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.846168 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.846129 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-data\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.846294 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.846255 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-credential-socket\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.846495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.846430 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-data\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.846495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.846465 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-socket\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.846670 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.846589 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-certs\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.846670 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.846625 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6748e7d-aa1d-417c-bc7d-071cc5033646-istiod-ca-cert\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.848228 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.848196 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-envoy\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.848749 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.848726 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-podinfo\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.853894 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.853866 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-token\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.853989 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.853944 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrcsq\" (UniqueName: \"kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-kube-api-access-rrcsq\") pod \"e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:07.902040 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:07.902004 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:08.051419 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:08.051390 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj"] Jul 22 19:25:08.053492 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:25:08.053455 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6748e7d_aa1d_417c_bc7d_071cc5033646.slice/crio-e38f493cd793f0da797369b8a9dbfd840b44c2df674c96fe4ad77a011c07f3fe WatchSource:0}: Error finding container e38f493cd793f0da797369b8a9dbfd840b44c2df674c96fe4ad77a011c07f3fe: Status 404 returned error can't find the container with id e38f493cd793f0da797369b8a9dbfd840b44c2df674c96fe4ad77a011c07f3fe Jul 22 19:25:08.055806 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:08.055765 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:25:08.055892 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:08.055853 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:25:08.055931 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:08.055897 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:25:08.316770 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:08.316731 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" event={"ID":"f6748e7d-aa1d-417c-bc7d-071cc5033646","Type":"ContainerStarted","Data":"3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e"} Jul 22 19:25:08.316912 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:08.316792 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" event={"ID":"f6748e7d-aa1d-417c-bc7d-071cc5033646","Type":"ContainerStarted","Data":"e38f493cd793f0da797369b8a9dbfd840b44c2df674c96fe4ad77a011c07f3fe"} Jul 22 19:25:08.340131 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:08.340081 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" podStartSLOduration=1.340062151 podStartE2EDuration="1.340062151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:25:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:25:08.338657037 +0000 UTC m=+2042.001231376" watchObservedRunningTime="2026-07-22 19:25:08.340062151 +0000 UTC m=+2042.002636491" Jul 22 19:25:08.908136 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:08.908103 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:09.910089 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:09.910060 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:10.326906 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:10.326819 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:10.327754 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:10.327734 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:11.947136 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:11.947112 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:25:12.086696 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.086668 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxpm2\" (UniqueName: \"kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-kube-api-access-lxpm2\") pod \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " Jul 22 19:25:12.086875 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.086701 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-token\") pod \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " Jul 22 19:25:12.086875 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.086738 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-credential-socket\") pod \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " Jul 22 19:25:12.086985 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.086881 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-envoy\") pod \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " Jul 22 19:25:12.086985 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.086944 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-certs\") pod \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " Jul 22 19:25:12.087094 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.086989 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-socket\") pod \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " Jul 22 19:25:12.087094 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.087018 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-data\") pod \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " Jul 22 19:25:12.087094 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.087069 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dbcbb26c-9d76-4994-bef4-cfdb64abca9a" (UID: "dbcbb26c-9d76-4994-bef4-cfdb64abca9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:12.087262 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.087060 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-podinfo\") pod \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " Jul 22 19:25:12.087262 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.087149 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istiod-ca-cert\") pod \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\" (UID: \"dbcbb26c-9d76-4994-bef4-cfdb64abca9a\") " Jul 22 19:25:12.087635 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.087439 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dbcbb26c-9d76-4994-bef4-cfdb64abca9a" (UID: "dbcbb26c-9d76-4994-bef4-cfdb64abca9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:12.087635 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.087455 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dbcbb26c-9d76-4994-bef4-cfdb64abca9a" (UID: "dbcbb26c-9d76-4994-bef4-cfdb64abca9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:12.087635 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.087567 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:12.088003 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.087977 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dbcbb26c-9d76-4994-bef4-cfdb64abca9a" (UID: "dbcbb26c-9d76-4994-bef4-cfdb64abca9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:25:12.088233 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.088209 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-data" (OuterVolumeSpecName: "istio-data") pod "dbcbb26c-9d76-4994-bef4-cfdb64abca9a" (UID: "dbcbb26c-9d76-4994-bef4-cfdb64abca9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:12.089425 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.089395 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dbcbb26c-9d76-4994-bef4-cfdb64abca9a" (UID: "dbcbb26c-9d76-4994-bef4-cfdb64abca9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:25:12.089759 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.089736 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-kube-api-access-lxpm2" (OuterVolumeSpecName: "kube-api-access-lxpm2") pod "dbcbb26c-9d76-4994-bef4-cfdb64abca9a" (UID: "dbcbb26c-9d76-4994-bef4-cfdb64abca9a"). InnerVolumeSpecName "kube-api-access-lxpm2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:25:12.090106 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.090081 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dbcbb26c-9d76-4994-bef4-cfdb64abca9a" (UID: "dbcbb26c-9d76-4994-bef4-cfdb64abca9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:12.090201 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.090152 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-token" (OuterVolumeSpecName: "istio-token") pod "dbcbb26c-9d76-4994-bef4-cfdb64abca9a" (UID: "dbcbb26c-9d76-4994-bef4-cfdb64abca9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:25:12.188147 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.188116 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxpm2\" (UniqueName: \"kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-kube-api-access-lxpm2\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:12.188147 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.188142 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:12.188147 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.188154 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:12.188378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.188162 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:12.188378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.188170 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:12.188378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.188177 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:12.188378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.188185 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:12.188378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.188193 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbcbb26c-9d76-4994-bef4-cfdb64abca9a-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:12.342251 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.342163 2584 generic.go:356] "Generic (PLEG): container finished" podID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerID="116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74" exitCode=0 Jul 22 19:25:12.342251 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.342209 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" Jul 22 19:25:12.342425 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.342251 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" event={"ID":"dbcbb26c-9d76-4994-bef4-cfdb64abca9a","Type":"ContainerDied","Data":"116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74"} Jul 22 19:25:12.342425 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.342327 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw" event={"ID":"dbcbb26c-9d76-4994-bef4-cfdb64abca9a","Type":"ContainerDied","Data":"5c33aea40e9e1f130f10d51cbeb3a48867bba4de333677b1b19c7d49fda4fe44"} Jul 22 19:25:12.342425 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.342352 2584 scope.go:117] "RemoveContainer" containerID="116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74" Jul 22 19:25:12.351911 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.351896 2584 scope.go:117] "RemoveContainer" containerID="4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a" Jul 22 19:25:12.359782 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.359763 2584 scope.go:117] "RemoveContainer" containerID="116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74" Jul 22 19:25:12.360017 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:25:12.359999 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74\": container with ID starting with 116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74 not found: ID does not exist" containerID="116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74" Jul 22 19:25:12.360079 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.360025 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74"} err="failed to get container status \"116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74\": rpc error: code = NotFound desc = could not find container \"116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74\": container with ID starting with 116e522c597ae7d8446d088cd5dfce42e50012883a054db47697a9193c6b6a74 not found: ID does not exist" Jul 22 19:25:12.360079 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.360047 2584 scope.go:117] "RemoveContainer" containerID="4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a" Jul 22 19:25:12.360275 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:25:12.360260 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a\": container with ID starting with 4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a not found: ID does not exist" containerID="4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a" Jul 22 19:25:12.360316 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.360278 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a"} err="failed to get container status \"4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a\": rpc error: code = NotFound desc = could not find container \"4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a\": container with ID starting with 4f8927fbe6008d77c9485ac643311252fb8d6b264ad67b40cb1f13f499d3627a not found: ID does not exist" Jul 22 19:25:12.365197 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.365176 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw"] Jul 22 19:25:12.370414 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.370392 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3df5cf99-openshift-default-f547f5cfc-xs7tw"] Jul 22 19:25:12.906863 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:12.906832 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" path="/var/lib/kubelet/pods/dbcbb26c-9d76-4994-bef4-cfdb64abca9a/volumes" Jul 22 19:25:23.400067 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:23.400000 2584 generic.go:356] "Generic (PLEG): container finished" podID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerID="3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e" exitCode=0 Jul 22 19:25:23.400067 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:23.400050 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" event={"ID":"f6748e7d-aa1d-417c-bc7d-071cc5033646","Type":"ContainerDied","Data":"3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e"} Jul 22 19:25:23.400482 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:23.400374 2584 scope.go:117] "RemoveContainer" containerID="3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e" Jul 22 19:25:23.407293 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:23.407254 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:25:23.407452 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:23.407346 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:25:23.407452 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:23.407384 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:25:24.410419 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:24.410385 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" event={"ID":"f6748e7d-aa1d-417c-bc7d-071cc5033646","Type":"ContainerStarted","Data":"ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822"} Jul 22 19:25:24.906123 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:24.906085 2584 patch_prober.go:28] interesting pod/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 22 19:25:24.906323 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:24.906150 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 22 19:25:24.909955 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:24.909929 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:25.327664 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:25.327587 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:25.903303 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:25.903261 2584 patch_prober.go:28] interesting pod/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 22 19:25:25.903848 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:25.903327 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 22 19:25:26.905017 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:26.904923 2584 patch_prober.go:28] interesting pod/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 22 19:25:26.905017 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:26.904990 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 22 19:25:27.902975 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:27.902927 2584 patch_prober.go:28] interesting pod/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 22 19:25:27.903178 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:27.902996 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 22 19:25:28.907594 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:28.907566 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:28.908555 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:28.908508 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:49.891034 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:49.890988 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj"] Jul 22 19:25:49.891657 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:49.891325 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" containerID="cri-o://ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822" gracePeriod=30 Jul 22 19:25:55.149999 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.149972 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:55.271340 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271254 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-envoy\") pod \"f6748e7d-aa1d-417c-bc7d-071cc5033646\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " Jul 22 19:25:55.271340 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271294 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-podinfo\") pod \"f6748e7d-aa1d-417c-bc7d-071cc5033646\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " Jul 22 19:25:55.271593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271346 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-data\") pod \"f6748e7d-aa1d-417c-bc7d-071cc5033646\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " Jul 22 19:25:55.271593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271397 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-credential-socket\") pod \"f6748e7d-aa1d-417c-bc7d-071cc5033646\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " Jul 22 19:25:55.271593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271434 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-certs\") pod \"f6748e7d-aa1d-417c-bc7d-071cc5033646\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " Jul 22 19:25:55.271593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271459 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-socket\") pod \"f6748e7d-aa1d-417c-bc7d-071cc5033646\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " Jul 22 19:25:55.271593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271481 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrcsq\" (UniqueName: \"kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-kube-api-access-rrcsq\") pod \"f6748e7d-aa1d-417c-bc7d-071cc5033646\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " Jul 22 19:25:55.271593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271505 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-token\") pod \"f6748e7d-aa1d-417c-bc7d-071cc5033646\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " Jul 22 19:25:55.271593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271581 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6748e7d-aa1d-417c-bc7d-071cc5033646-istiod-ca-cert\") pod \"f6748e7d-aa1d-417c-bc7d-071cc5033646\" (UID: \"f6748e7d-aa1d-417c-bc7d-071cc5033646\") " Jul 22 19:25:55.271937 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271699 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f6748e7d-aa1d-417c-bc7d-071cc5033646" (UID: "f6748e7d-aa1d-417c-bc7d-071cc5033646"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:55.271937 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271713 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f6748e7d-aa1d-417c-bc7d-071cc5033646" (UID: "f6748e7d-aa1d-417c-bc7d-071cc5033646"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:55.272075 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271943 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.272075 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.271963 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.272177 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.272120 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6748e7d-aa1d-417c-bc7d-071cc5033646-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f6748e7d-aa1d-417c-bc7d-071cc5033646" (UID: "f6748e7d-aa1d-417c-bc7d-071cc5033646"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:25:55.272275 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.272251 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f6748e7d-aa1d-417c-bc7d-071cc5033646" (UID: "f6748e7d-aa1d-417c-bc7d-071cc5033646"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:55.272474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.272442 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-data" (OuterVolumeSpecName: "istio-data") pod "f6748e7d-aa1d-417c-bc7d-071cc5033646" (UID: "f6748e7d-aa1d-417c-bc7d-071cc5033646"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:55.273914 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.273886 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f6748e7d-aa1d-417c-bc7d-071cc5033646" (UID: "f6748e7d-aa1d-417c-bc7d-071cc5033646"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:25:55.274032 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.274016 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6748e7d-aa1d-417c-bc7d-071cc5033646" (UID: "f6748e7d-aa1d-417c-bc7d-071cc5033646"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:25:55.274097 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.274062 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f6748e7d-aa1d-417c-bc7d-071cc5033646" (UID: "f6748e7d-aa1d-417c-bc7d-071cc5033646"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:25:55.274335 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.274305 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-kube-api-access-rrcsq" (OuterVolumeSpecName: "kube-api-access-rrcsq") pod "f6748e7d-aa1d-417c-bc7d-071cc5033646" (UID: "f6748e7d-aa1d-417c-bc7d-071cc5033646"). InnerVolumeSpecName "kube-api-access-rrcsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:25:55.372810 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.372772 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.372810 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.372804 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.373018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.372818 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrcsq\" (UniqueName: \"kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-kube-api-access-rrcsq\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.373018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.372831 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.373018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.372842 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6748e7d-aa1d-417c-bc7d-071cc5033646-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.373018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.372854 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.373018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.372864 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6748e7d-aa1d-417c-bc7d-071cc5033646-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:25:55.575930 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.575836 2584 generic.go:356] "Generic (PLEG): container finished" podID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerID="ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822" exitCode=0 Jul 22 19:25:55.576079 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.575984 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" Jul 22 19:25:55.576079 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.576010 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" event={"ID":"f6748e7d-aa1d-417c-bc7d-071cc5033646","Type":"ContainerDied","Data":"ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822"} Jul 22 19:25:55.576172 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.576093 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj" event={"ID":"f6748e7d-aa1d-417c-bc7d-071cc5033646","Type":"ContainerDied","Data":"e38f493cd793f0da797369b8a9dbfd840b44c2df674c96fe4ad77a011c07f3fe"} Jul 22 19:25:55.576172 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.576111 2584 scope.go:117] "RemoveContainer" containerID="ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822" Jul 22 19:25:55.586043 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.586016 2584 scope.go:117] "RemoveContainer" containerID="3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e" Jul 22 19:25:55.594217 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.594198 2584 scope.go:117] "RemoveContainer" containerID="ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822" Jul 22 19:25:55.594467 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:25:55.594447 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822\": container with ID starting with ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822 not found: ID does not exist" containerID="ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822" Jul 22 19:25:55.594548 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.594476 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822"} err="failed to get container status \"ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822\": rpc error: code = NotFound desc = could not find container \"ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822\": container with ID starting with ebb3a21431db59b9acd4e7cc6975f02cf5c60a99e92507ef992420f8174d0822 not found: ID does not exist" Jul 22 19:25:55.594548 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.594493 2584 scope.go:117] "RemoveContainer" containerID="3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e" Jul 22 19:25:55.595152 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:25:55.594826 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e\": container with ID starting with 3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e not found: ID does not exist" containerID="3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e" Jul 22 19:25:55.595152 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.594860 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e"} err="failed to get container status \"3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e\": rpc error: code = NotFound desc = could not find container \"3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e\": container with ID starting with 3fe4cfeff5c7636a7d07ce4f14c4fb49a1df8796d9cc396391bd58a3c0d1397e not found: ID does not exist" Jul 22 19:25:55.598279 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.598253 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj"] Jul 22 19:25:55.601745 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:55.601720 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-bb81af95-openshift-default-7996574d5f-5b9kj"] Jul 22 19:25:56.906899 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:25:56.906866 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" path="/var/lib/kubelet/pods/f6748e7d-aa1d-417c-bc7d-071cc5033646/volumes" Jul 22 19:26:09.300631 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:26:09.300600 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:26:09.306574 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:26:09.306546 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:27:31.558551 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558461 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc"] Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558805 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558816 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="061405a7-f75d-48b0-9f1e-e8158f9bfc2e" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558831 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558837 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558845 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558852 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558865 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558874 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558887 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558892 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558958 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558967 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" Jul 22 19:27:31.559155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.558975 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbcbb26c-9d76-4994-bef4-cfdb64abca9a" containerName="istio-proxy" Jul 22 19:27:31.561694 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.561678 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.564140 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.564119 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-890e74a3-openshift-default-dockercfg-bmc5k\"" Jul 22 19:27:31.581051 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.581024 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc"] Jul 22 19:27:31.699258 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.699218 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-socket\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.699258 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.699258 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-envoy\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.699544 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.699279 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-data\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.699544 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.699321 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-podinfo\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.699544 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.699428 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istiod-ca-cert\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.699544 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.699478 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-certs\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.699757 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.699608 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xdc4\" (UniqueName: \"kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-kube-api-access-5xdc4\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.699757 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.699660 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-token\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.699757 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.699696 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-credential-socket\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.800770 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.800724 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-certs\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.800968 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.800815 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xdc4\" (UniqueName: \"kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-kube-api-access-5xdc4\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.800968 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.800847 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-token\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.800968 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.800865 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-credential-socket\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.800968 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.800914 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-socket\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.800968 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.800945 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-envoy\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.801253 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.800971 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-data\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.801253 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.800997 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-podinfo\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.801253 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.801033 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istiod-ca-cert\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.801427 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.801294 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-credential-socket\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.801427 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.801358 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-data\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.801577 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.801482 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-certs\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.801577 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.801560 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-socket\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.801783 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.801765 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istiod-ca-cert\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.803718 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.803694 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-podinfo\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.803718 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.803702 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-envoy\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.808794 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.808737 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-token\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.809186 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.809163 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xdc4\" (UniqueName: \"kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-kube-api-access-5xdc4\") pod \"e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:31.872883 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:31.872850 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:32.016852 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:32.016820 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc"] Jul 22 19:27:32.017946 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:27:32.017898 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e5389b0_fce8_4d82_aa4d_756150cb1d86.slice/crio-28634d54246ec6250b430ea9d6317608220dba3fb02df3e17ef904fba0bd9c08 WatchSource:0}: Error finding container 28634d54246ec6250b430ea9d6317608220dba3fb02df3e17ef904fba0bd9c08: Status 404 returned error can't find the container with id 28634d54246ec6250b430ea9d6317608220dba3fb02df3e17ef904fba0bd9c08 Jul 22 19:27:32.020158 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:32.020124 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:27:32.020256 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:32.020187 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:27:32.020256 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:32.020216 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:27:32.071636 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:32.070956 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" event={"ID":"9e5389b0-fce8-4d82-aa4d-756150cb1d86","Type":"ContainerStarted","Data":"28634d54246ec6250b430ea9d6317608220dba3fb02df3e17ef904fba0bd9c08"} Jul 22 19:27:33.077807 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:33.077762 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" event={"ID":"9e5389b0-fce8-4d82-aa4d-756150cb1d86","Type":"ContainerStarted","Data":"f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6"} Jul 22 19:27:33.109583 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:33.109531 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" podStartSLOduration=2.109503743 podStartE2EDuration="2.109503743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:27:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:27:33.106561988 +0000 UTC m=+2186.769136319" watchObservedRunningTime="2026-07-22 19:27:33.109503743 +0000 UTC m=+2186.772078081" Jul 22 19:27:33.873069 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:33.873029 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:33.885061 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:33.885025 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:34.084177 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:34.084140 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:34.085197 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:34.085174 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:45.149003 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:45.148968 2584 generic.go:356] "Generic (PLEG): container finished" podID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerID="f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6" exitCode=0 Jul 22 19:27:45.149545 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:45.149050 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" event={"ID":"9e5389b0-fce8-4d82-aa4d-756150cb1d86","Type":"ContainerDied","Data":"f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6"} Jul 22 19:27:45.149625 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:45.149576 2584 scope.go:117] "RemoveContainer" containerID="f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6" Jul 22 19:27:45.150756 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:45.150730 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:27:45.150837 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:45.150799 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:27:45.150837 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:45.150832 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:27:46.156704 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:46.156667 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" event={"ID":"9e5389b0-fce8-4d82-aa4d-756150cb1d86","Type":"ContainerStarted","Data":"d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203"} Jul 22 19:27:46.873874 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:46.873841 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:46.874992 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:46.874971 2584 patch_prober.go:28] interesting pod/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 22 19:27:46.875107 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:46.875026 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 22 19:27:47.873455 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:47.873417 2584 patch_prober.go:28] interesting pod/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 22 19:27:47.873873 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:47.873472 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 22 19:27:48.873373 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:48.873332 2584 patch_prober.go:28] interesting pod/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 22 19:27:48.873588 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:48.873406 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 22 19:27:49.084999 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:49.084957 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:49.874011 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:49.873978 2584 patch_prober.go:28] interesting pod/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 22 19:27:49.874408 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:49.874031 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 22 19:27:50.877068 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:50.877037 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:27:50.878009 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:27:50.877984 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:28:07.837600 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:07.837564 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc"] Jul 22 19:28:07.838205 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:07.837809 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" containerID="cri-o://d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203" gracePeriod=30 Jul 22 19:28:13.088005 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.087981 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:28:13.176177 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176103 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-data\") pod \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " Jul 22 19:28:13.176177 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176144 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-credential-socket\") pod \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " Jul 22 19:28:13.176177 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176167 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-envoy\") pod \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " Jul 22 19:28:13.176416 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176208 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-podinfo\") pod \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " Jul 22 19:28:13.176416 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176224 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istiod-ca-cert\") pod \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " Jul 22 19:28:13.176416 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176243 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-token\") pod \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " Jul 22 19:28:13.176416 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176269 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-certs\") pod \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " Jul 22 19:28:13.176416 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176410 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xdc4\" (UniqueName: \"kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-kube-api-access-5xdc4\") pod \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " Jul 22 19:28:13.176780 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176421 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e5389b0-fce8-4d82-aa4d-756150cb1d86" (UID: "9e5389b0-fce8-4d82-aa4d-756150cb1d86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:28:13.176780 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176453 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-socket\") pod \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\" (UID: \"9e5389b0-fce8-4d82-aa4d-756150cb1d86\") " Jul 22 19:28:13.176780 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176770 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:28:13.176944 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176795 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e5389b0-fce8-4d82-aa4d-756150cb1d86" (UID: "9e5389b0-fce8-4d82-aa4d-756150cb1d86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:28:13.177001 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176980 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e5389b0-fce8-4d82-aa4d-756150cb1d86" (UID: "9e5389b0-fce8-4d82-aa4d-756150cb1d86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:28:13.177057 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.176988 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e5389b0-fce8-4d82-aa4d-756150cb1d86" (UID: "9e5389b0-fce8-4d82-aa4d-756150cb1d86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:28:13.177122 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.177077 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e5389b0-fce8-4d82-aa4d-756150cb1d86" (UID: "9e5389b0-fce8-4d82-aa4d-756150cb1d86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:28:13.178702 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.178671 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e5389b0-fce8-4d82-aa4d-756150cb1d86" (UID: "9e5389b0-fce8-4d82-aa4d-756150cb1d86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:28:13.178958 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.178935 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-kube-api-access-5xdc4" (OuterVolumeSpecName: "kube-api-access-5xdc4") pod "9e5389b0-fce8-4d82-aa4d-756150cb1d86" (UID: "9e5389b0-fce8-4d82-aa4d-756150cb1d86"). InnerVolumeSpecName "kube-api-access-5xdc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:28:13.179120 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.179101 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e5389b0-fce8-4d82-aa4d-756150cb1d86" (UID: "9e5389b0-fce8-4d82-aa4d-756150cb1d86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:28:13.179185 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.179168 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e5389b0-fce8-4d82-aa4d-756150cb1d86" (UID: "9e5389b0-fce8-4d82-aa4d-756150cb1d86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:28:13.277491 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.277459 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:28:13.277491 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.277491 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:28:13.277652 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.277505 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:28:13.277652 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.277536 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:28:13.277652 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.277552 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xdc4\" (UniqueName: \"kubernetes.io/projected/9e5389b0-fce8-4d82-aa4d-756150cb1d86-kube-api-access-5xdc4\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:28:13.277652 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.277566 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:28:13.277652 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.277579 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:28:13.277652 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.277587 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e5389b0-fce8-4d82-aa4d-756150cb1d86-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:28:13.298667 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.298641 2584 generic.go:356] "Generic (PLEG): container finished" podID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerID="d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203" exitCode=0 Jul 22 19:28:13.298820 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.298694 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" Jul 22 19:28:13.298820 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.298718 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" event={"ID":"9e5389b0-fce8-4d82-aa4d-756150cb1d86","Type":"ContainerDied","Data":"d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203"} Jul 22 19:28:13.298820 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.298763 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc" event={"ID":"9e5389b0-fce8-4d82-aa4d-756150cb1d86","Type":"ContainerDied","Data":"28634d54246ec6250b430ea9d6317608220dba3fb02df3e17ef904fba0bd9c08"} Jul 22 19:28:13.298820 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.298787 2584 scope.go:117] "RemoveContainer" containerID="d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203" Jul 22 19:28:13.309437 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.309418 2584 scope.go:117] "RemoveContainer" containerID="f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6" Jul 22 19:28:13.318248 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.318231 2584 scope.go:117] "RemoveContainer" containerID="d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203" Jul 22 19:28:13.318541 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:28:13.318494 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203\": container with ID starting with d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203 not found: ID does not exist" containerID="d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203" Jul 22 19:28:13.318593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.318544 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203"} err="failed to get container status \"d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203\": rpc error: code = NotFound desc = could not find container \"d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203\": container with ID starting with d375dc76bbb293319be12c91029c4713d71c903d6d58b3887653715d4bea2203 not found: ID does not exist" Jul 22 19:28:13.318593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.318566 2584 scope.go:117] "RemoveContainer" containerID="f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6" Jul 22 19:28:13.318804 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:28:13.318785 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6\": container with ID starting with f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6 not found: ID does not exist" containerID="f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6" Jul 22 19:28:13.318866 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.318807 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6"} err="failed to get container status \"f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6\": rpc error: code = NotFound desc = could not find container \"f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6\": container with ID starting with f62c18328025013f45418dcc5fa7a112979c59cd9d4bd2efd8ff5e94a4496bd6 not found: ID does not exist" Jul 22 19:28:13.322941 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.322920 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc"] Jul 22 19:28:13.327249 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:13.327229 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-890e74a3-openshift-default-55b4fb8d6c-62zvc"] Jul 22 19:28:14.907573 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:14.907540 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" path="/var/lib/kubelet/pods/9e5389b0-fce8-4d82-aa4d-756150cb1d86/volumes" Jul 22 19:28:27.245956 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.245877 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x"] Jul 22 19:28:27.246386 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.246370 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" Jul 22 19:28:27.246438 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.246389 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" Jul 22 19:28:27.246438 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.246414 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" Jul 22 19:28:27.246438 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.246423 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" Jul 22 19:28:27.246568 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.246511 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" Jul 22 19:28:27.246568 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.246542 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6748e7d-aa1d-417c-bc7d-071cc5033646" containerName="istio-proxy" Jul 22 19:28:27.246568 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.246551 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e5389b0-fce8-4d82-aa4d-756150cb1d86" containerName="istio-proxy" Jul 22 19:28:27.251168 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.251146 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.253915 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.253893 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5308e856-openshift-default-dockercfg-2hhcs\"" Jul 22 19:28:27.261491 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.261469 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x"] Jul 22 19:28:27.393369 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.393329 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-data\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.393589 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.393396 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-socket\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.393589 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.393458 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec97c5ed-232f-4452-97d7-f61e25f51d14-istiod-ca-cert\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.393589 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.393510 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-certs\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.393764 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.393587 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-podinfo\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.393764 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.393622 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-credential-socket\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.393764 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.393670 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-envoy\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.393764 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.393692 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-token\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.393764 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.393721 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrcxt\" (UniqueName: \"kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-kube-api-access-mrcxt\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.494394 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494359 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-certs\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.494394 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494396 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-podinfo\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.494663 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494420 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-credential-socket\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.494663 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494449 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-envoy\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.494663 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494473 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-token\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.494663 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494496 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrcxt\" (UniqueName: \"kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-kube-api-access-mrcxt\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.494867 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494769 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-data\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.494867 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494821 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-socket\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.494867 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494835 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-certs\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.495026 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494885 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-credential-socket\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.495026 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.494939 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec97c5ed-232f-4452-97d7-f61e25f51d14-istiod-ca-cert\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.495142 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.495076 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-socket\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.495202 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.495177 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-data\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.495908 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.495882 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec97c5ed-232f-4452-97d7-f61e25f51d14-istiod-ca-cert\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.496982 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.496938 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-envoy\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.497132 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.497110 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-podinfo\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.509200 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.509176 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-token\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.518026 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.518006 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrcxt\" (UniqueName: \"kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-kube-api-access-mrcxt\") pod \"e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.563940 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.563918 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:27.708209 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.708180 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x"] Jul 22 19:28:27.709928 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:28:27.709895 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec97c5ed_232f_4452_97d7_f61e25f51d14.slice/crio-2870c36f84b6ae177f6678c01e6464b088971006658e61c6ea239ed2cc0cf3d8 WatchSource:0}: Error finding container 2870c36f84b6ae177f6678c01e6464b088971006658e61c6ea239ed2cc0cf3d8: Status 404 returned error can't find the container with id 2870c36f84b6ae177f6678c01e6464b088971006658e61c6ea239ed2cc0cf3d8 Jul 22 19:28:27.712068 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.712034 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:28:27.712133 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.712104 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:28:27.712172 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:27.712131 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:28:28.378128 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:28.378094 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" event={"ID":"ec97c5ed-232f-4452-97d7-f61e25f51d14","Type":"ContainerStarted","Data":"f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841"} Jul 22 19:28:28.378548 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:28.378147 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" event={"ID":"ec97c5ed-232f-4452-97d7-f61e25f51d14","Type":"ContainerStarted","Data":"2870c36f84b6ae177f6678c01e6464b088971006658e61c6ea239ed2cc0cf3d8"} Jul 22 19:28:28.399221 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:28.399169 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" podStartSLOduration=1.399153154 podStartE2EDuration="1.399153154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:28:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:28:28.398668628 +0000 UTC m=+2242.061242989" watchObservedRunningTime="2026-07-22 19:28:28.399153154 +0000 UTC m=+2242.061727492" Jul 22 19:28:28.564207 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:28.564168 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:28.569481 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:28.569454 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:29.383403 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:29.383371 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:29.384501 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:29.384482 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:44.385709 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:44.385678 2584 patch_prober.go:28] interesting pod/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 22 19:28:44.386092 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:44.385753 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 22 19:28:44.465290 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:44.465257 2584 generic.go:356] "Generic (PLEG): container finished" podID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerID="f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841" exitCode=0 Jul 22 19:28:44.465421 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:44.465340 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" event={"ID":"ec97c5ed-232f-4452-97d7-f61e25f51d14","Type":"ContainerDied","Data":"f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841"} Jul 22 19:28:44.465787 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:44.465765 2584 scope.go:117] "RemoveContainer" containerID="f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841" Jul 22 19:28:44.466817 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:44.466790 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:28:44.466894 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:44.466858 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:28:44.466937 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:44.466897 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:28:45.472441 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:45.472404 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" event={"ID":"ec97c5ed-232f-4452-97d7-f61e25f51d14","Type":"ContainerStarted","Data":"5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d"} Jul 22 19:28:45.565122 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:45.565090 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:45.565273 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:45.565249 2584 patch_prober.go:28] interesting pod/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 22 19:28:45.565318 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:45.565294 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 22 19:28:46.564351 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:46.564312 2584 patch_prober.go:28] interesting pod/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 22 19:28:46.564810 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:46.564368 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 22 19:28:47.564535 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:47.564479 2584 patch_prober.go:28] interesting pod/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 22 19:28:47.564954 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:47.564563 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 22 19:28:48.565037 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:48.565007 2584 patch_prober.go:28] interesting pod/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 22 19:28:48.565451 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:48.565062 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 22 19:28:49.564483 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:49.564448 2584 patch_prober.go:28] interesting pod/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 22 19:28:49.564680 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:49.564503 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 22 19:28:50.568688 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:50.568657 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:50.569089 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:50.569018 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:28:50.569652 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:28:50.569633 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:29:13.203067 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:13.203026 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w"] Jul 22 19:29:13.205817 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:13.205798 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w" Jul 22 19:29:13.208372 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:13.208349 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\"" Jul 22 19:29:13.214389 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:13.214364 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w"] Jul 22 19:29:13.319764 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:13.319725 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0068a8bb-52f7-48b9-bc24-629afcee6c75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5308e856-rnw7w\" (UID: \"0068a8bb-52f7-48b9-bc24-629afcee6c75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w" Jul 22 19:29:13.421057 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:13.421021 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0068a8bb-52f7-48b9-bc24-629afcee6c75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5308e856-rnw7w\" (UID: \"0068a8bb-52f7-48b9-bc24-629afcee6c75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w" Jul 22 19:29:13.421745 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:13.421715 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0068a8bb-52f7-48b9-bc24-629afcee6c75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5308e856-rnw7w\" (UID: \"0068a8bb-52f7-48b9-bc24-629afcee6c75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w" Jul 22 19:29:13.520259 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:13.520174 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w" Jul 22 19:29:13.652919 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:13.652872 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w"] Jul 22 19:29:13.654810 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:29:13.654778 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0068a8bb_52f7_48b9_bc24_629afcee6c75.slice/crio-f82eb3aeacff2cdd86167756fc6059e54cc30b0e2c5b4620c902d8e24f814034 WatchSource:0}: Error finding container f82eb3aeacff2cdd86167756fc6059e54cc30b0e2c5b4620c902d8e24f814034: Status 404 returned error can't find the container with id f82eb3aeacff2cdd86167756fc6059e54cc30b0e2c5b4620c902d8e24f814034 Jul 22 19:29:13.656777 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:13.656755 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 19:29:14.632472 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:14.632430 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w" event={"ID":"0068a8bb-52f7-48b9-bc24-629afcee6c75","Type":"ContainerStarted","Data":"f82eb3aeacff2cdd86167756fc6059e54cc30b0e2c5b4620c902d8e24f814034"} Jul 22 19:29:16.644814 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:16.644781 2584 generic.go:356] "Generic (PLEG): container finished" podID="0068a8bb-52f7-48b9-bc24-629afcee6c75" containerID="f0be23f0a39d28331953c01f23568564939af2712e27fae83749239faf927d5e" exitCode=0 Jul 22 19:29:16.645238 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:16.644827 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w" event={"ID":"0068a8bb-52f7-48b9-bc24-629afcee6c75","Type":"ContainerDied","Data":"f0be23f0a39d28331953c01f23568564939af2712e27fae83749239faf927d5e"} Jul 22 19:29:17.785227 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:17.785196 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w" Jul 22 19:29:17.965502 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:17.965419 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0068a8bb-52f7-48b9-bc24-629afcee6c75-maas-api-service-ca\") pod \"0068a8bb-52f7-48b9-bc24-629afcee6c75\" (UID: \"0068a8bb-52f7-48b9-bc24-629afcee6c75\") " Jul 22 19:29:17.965861 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:17.965838 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0068a8bb-52f7-48b9-bc24-629afcee6c75-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0068a8bb-52f7-48b9-bc24-629afcee6c75" (UID: "0068a8bb-52f7-48b9-bc24-629afcee6c75"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:29:18.066490 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:18.066443 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0068a8bb-52f7-48b9-bc24-629afcee6c75-maas-api-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:29:18.657746 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:18.657703 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w" event={"ID":"0068a8bb-52f7-48b9-bc24-629afcee6c75","Type":"ContainerDied","Data":"f82eb3aeacff2cdd86167756fc6059e54cc30b0e2c5b4620c902d8e24f814034"} Jul 22 19:29:18.657918 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:18.657763 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f82eb3aeacff2cdd86167756fc6059e54cc30b0e2c5b4620c902d8e24f814034" Jul 22 19:29:18.657918 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:18.657714 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w" Jul 22 19:29:33.270219 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:33.270183 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w"] Jul 22 19:29:33.272645 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:33.272621 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5308e856-rnw7w"] Jul 22 19:29:34.906191 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:29:34.906156 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0068a8bb-52f7-48b9-bc24-629afcee6c75" path="/var/lib/kubelet/pods/0068a8bb-52f7-48b9-bc24-629afcee6c75/volumes" Jul 22 19:30:23.142134 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:23.142058 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x"] Jul 22 19:30:23.142666 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:23.142318 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" containerID="cri-o://5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d" gracePeriod=30 Jul 22 19:30:28.389888 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.389859 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:30:28.487201 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487110 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-token\") pod \"ec97c5ed-232f-4452-97d7-f61e25f51d14\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " Jul 22 19:30:28.487201 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487157 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-socket\") pod \"ec97c5ed-232f-4452-97d7-f61e25f51d14\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " Jul 22 19:30:28.487201 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487176 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-envoy\") pod \"ec97c5ed-232f-4452-97d7-f61e25f51d14\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " Jul 22 19:30:28.487475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487238 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-data\") pod \"ec97c5ed-232f-4452-97d7-f61e25f51d14\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " Jul 22 19:30:28.487475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487258 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-podinfo\") pod \"ec97c5ed-232f-4452-97d7-f61e25f51d14\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " Jul 22 19:30:28.487475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487292 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-certs\") pod \"ec97c5ed-232f-4452-97d7-f61e25f51d14\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " Jul 22 19:30:28.487475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487343 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-credential-socket\") pod \"ec97c5ed-232f-4452-97d7-f61e25f51d14\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " Jul 22 19:30:28.487475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487407 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec97c5ed-232f-4452-97d7-f61e25f51d14-istiod-ca-cert\") pod \"ec97c5ed-232f-4452-97d7-f61e25f51d14\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " Jul 22 19:30:28.487475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487428 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec97c5ed-232f-4452-97d7-f61e25f51d14" (UID: "ec97c5ed-232f-4452-97d7-f61e25f51d14"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:30:28.487475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487441 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrcxt\" (UniqueName: \"kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-kube-api-access-mrcxt\") pod \"ec97c5ed-232f-4452-97d7-f61e25f51d14\" (UID: \"ec97c5ed-232f-4452-97d7-f61e25f51d14\") " Jul 22 19:30:28.487873 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487650 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec97c5ed-232f-4452-97d7-f61e25f51d14" (UID: "ec97c5ed-232f-4452-97d7-f61e25f51d14"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:30:28.487931 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487868 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:30:28.487931 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487890 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:30:28.488035 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.487952 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec97c5ed-232f-4452-97d7-f61e25f51d14-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec97c5ed-232f-4452-97d7-f61e25f51d14" (UID: "ec97c5ed-232f-4452-97d7-f61e25f51d14"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:30:28.488133 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.488104 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec97c5ed-232f-4452-97d7-f61e25f51d14" (UID: "ec97c5ed-232f-4452-97d7-f61e25f51d14"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:30:28.488244 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.488218 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec97c5ed-232f-4452-97d7-f61e25f51d14" (UID: "ec97c5ed-232f-4452-97d7-f61e25f51d14"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:30:28.489656 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.489634 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec97c5ed-232f-4452-97d7-f61e25f51d14" (UID: "ec97c5ed-232f-4452-97d7-f61e25f51d14"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:30:28.490156 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.490113 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec97c5ed-232f-4452-97d7-f61e25f51d14" (UID: "ec97c5ed-232f-4452-97d7-f61e25f51d14"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:30:28.490242 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.490173 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec97c5ed-232f-4452-97d7-f61e25f51d14" (UID: "ec97c5ed-232f-4452-97d7-f61e25f51d14"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:30:28.490242 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.490210 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-kube-api-access-mrcxt" (OuterVolumeSpecName: "kube-api-access-mrcxt") pod "ec97c5ed-232f-4452-97d7-f61e25f51d14" (UID: "ec97c5ed-232f-4452-97d7-f61e25f51d14"). InnerVolumeSpecName "kube-api-access-mrcxt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:30:28.588329 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.588298 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:30:28.588329 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.588326 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:30:28.588560 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.588337 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:30:28.588560 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.588347 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec97c5ed-232f-4452-97d7-f61e25f51d14-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:30:28.588560 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.588356 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrcxt\" (UniqueName: \"kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-kube-api-access-mrcxt\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:30:28.588560 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.588366 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:30:28.588560 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:28.588374 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec97c5ed-232f-4452-97d7-f61e25f51d14-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:30:29.019396 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.019363 2584 generic.go:356] "Generic (PLEG): container finished" podID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerID="5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d" exitCode=0 Jul 22 19:30:29.019567 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.019414 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" Jul 22 19:30:29.019567 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.019437 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" event={"ID":"ec97c5ed-232f-4452-97d7-f61e25f51d14","Type":"ContainerDied","Data":"5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d"} Jul 22 19:30:29.019567 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.019491 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x" event={"ID":"ec97c5ed-232f-4452-97d7-f61e25f51d14","Type":"ContainerDied","Data":"2870c36f84b6ae177f6678c01e6464b088971006658e61c6ea239ed2cc0cf3d8"} Jul 22 19:30:29.019567 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.019528 2584 scope.go:117] "RemoveContainer" containerID="5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d" Jul 22 19:30:29.029098 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.029078 2584 scope.go:117] "RemoveContainer" containerID="f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841" Jul 22 19:30:29.038134 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.038117 2584 scope.go:117] "RemoveContainer" containerID="5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d" Jul 22 19:30:29.038446 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:30:29.038425 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d\": container with ID starting with 5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d not found: ID does not exist" containerID="5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d" Jul 22 19:30:29.038548 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.038454 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d"} err="failed to get container status \"5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d\": rpc error: code = NotFound desc = could not find container \"5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d\": container with ID starting with 5aede8594de5da3ff8a71de6d41b534bd997eadb55578b56f6b6a74b47e3bc9d not found: ID does not exist" Jul 22 19:30:29.038548 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.038473 2584 scope.go:117] "RemoveContainer" containerID="f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841" Jul 22 19:30:29.038782 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:30:29.038761 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841\": container with ID starting with f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841 not found: ID does not exist" containerID="f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841" Jul 22 19:30:29.038879 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.038784 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841"} err="failed to get container status \"f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841\": rpc error: code = NotFound desc = could not find container \"f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841\": container with ID starting with f6b35db64c4cf7c26bd1ee604f427cb3231534e20edd6461c986359a85287841 not found: ID does not exist" Jul 22 19:30:29.040095 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.040074 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x"] Jul 22 19:30:29.043172 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:29.043148 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5308e856-openshift-default-9ffc9fdbc-sgh4x"] Jul 22 19:30:30.906697 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:30:30.906661 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" path="/var/lib/kubelet/pods/ec97c5ed-232f-4452-97d7-f61e25f51d14/volumes" Jul 22 19:31:09.336052 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:09.336021 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:31:09.348394 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:09.348363 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:31:17.616627 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.616591 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8"] Jul 22 19:31:17.617102 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.617084 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0068a8bb-52f7-48b9-bc24-629afcee6c75" containerName="revoke-keys" Jul 22 19:31:17.617175 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.617106 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0068a8bb-52f7-48b9-bc24-629afcee6c75" containerName="revoke-keys" Jul 22 19:31:17.617175 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.617127 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" Jul 22 19:31:17.617175 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.617136 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" Jul 22 19:31:17.617175 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.617156 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" Jul 22 19:31:17.617175 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.617165 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" Jul 22 19:31:17.617429 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.617236 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0068a8bb-52f7-48b9-bc24-629afcee6c75" containerName="revoke-keys" Jul 22 19:31:17.617429 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.617244 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" Jul 22 19:31:17.617429 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.617250 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec97c5ed-232f-4452-97d7-f61e25f51d14" containerName="istio-proxy" Jul 22 19:31:17.620528 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.620495 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.623156 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.623138 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-9c16c7-openshift-default-dockercfg-gqk5t\"" Jul 22 19:31:17.631636 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.631596 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8"] Jul 22 19:31:17.719447 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.719413 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-istio-token\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.719599 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.719457 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b15220-e472-4d30-9157-465c8073f513-istio-podinfo\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.719599 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.719476 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-socket\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.719599 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.719507 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbqxh\" (UniqueName: \"kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-kube-api-access-cbqxh\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.719599 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.719550 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-certs\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.719599 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.719564 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-data\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.719599 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.719590 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-envoy\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.719827 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.719627 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-credential-socket\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.719827 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.719670 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b15220-e472-4d30-9157-465c8073f513-istiod-ca-cert\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.820416 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820378 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-credential-socket\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.820416 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820414 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b15220-e472-4d30-9157-465c8073f513-istiod-ca-cert\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.820711 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820445 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-istio-token\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.820711 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820469 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b15220-e472-4d30-9157-465c8073f513-istio-podinfo\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.820711 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820492 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-socket\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.820711 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820676 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbqxh\" (UniqueName: \"kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-kube-api-access-cbqxh\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.820912 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820739 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-certs\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.820912 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820768 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-data\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.820912 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820824 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-envoy\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.821052 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820917 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-socket\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.821052 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.820964 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-credential-socket\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.821213 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.821192 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-certs\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.821267 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.821242 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-data\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.821358 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.821335 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b15220-e472-4d30-9157-465c8073f513-istiod-ca-cert\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.823134 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.823118 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-envoy\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.823450 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.823429 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b15220-e472-4d30-9157-465c8073f513-istio-podinfo\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.828872 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.828847 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbqxh\" (UniqueName: \"kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-kube-api-access-cbqxh\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.828959 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.828885 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-istio-token\") pod \"e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:17.931767 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:17.931685 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:18.069497 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:18.069470 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8"] Jul 22 19:31:18.071387 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:31:18.071358 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b15220_e472_4d30_9157_465c8073f513.slice/crio-35655ffafb6885d5b07809b50ddbcde2b6fa496b5bbf7e480a8abe7a28e2e02c WatchSource:0}: Error finding container 35655ffafb6885d5b07809b50ddbcde2b6fa496b5bbf7e480a8abe7a28e2e02c: Status 404 returned error can't find the container with id 35655ffafb6885d5b07809b50ddbcde2b6fa496b5bbf7e480a8abe7a28e2e02c Jul 22 19:31:18.073640 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:18.073609 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:18.073736 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:18.073676 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:18.073736 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:18.073706 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:18.280399 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:18.280301 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" event={"ID":"71b15220-e472-4d30-9157-465c8073f513","Type":"ContainerStarted","Data":"fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8"} Jul 22 19:31:18.280399 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:18.280359 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" event={"ID":"71b15220-e472-4d30-9157-465c8073f513","Type":"ContainerStarted","Data":"35655ffafb6885d5b07809b50ddbcde2b6fa496b5bbf7e480a8abe7a28e2e02c"} Jul 22 19:31:18.301043 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:18.300984 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" podStartSLOduration=1.300969406 podStartE2EDuration="1.300969406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:31:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:31:18.298203845 +0000 UTC m=+2411.960778197" watchObservedRunningTime="2026-07-22 19:31:18.300969406 +0000 UTC m=+2411.963543744" Jul 22 19:31:18.932191 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:18.932160 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:19.936885 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:19.936851 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:20.291555 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:20.291459 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:20.292408 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:20.292383 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:25.293051 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.293016 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj"] Jul 22 19:31:25.296980 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.296960 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.299862 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.299842 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-85c487-openshift-default-dockercfg-cp768\"" Jul 22 19:31:25.310034 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.310010 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj"] Jul 22 19:31:25.398971 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.398933 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-data\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.399256 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.399228 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-credential-socket\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.399395 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.399379 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9891e388-585d-4aad-8757-f0c6ea6536e8-istiod-ca-cert\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.399614 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.399594 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-token\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.399799 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.399782 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzldr\" (UniqueName: \"kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-kube-api-access-zzldr\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.399969 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.399950 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-certs\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.400087 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.400072 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-envoy\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.400198 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.400183 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-podinfo\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.400356 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.400342 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-socket\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.501787 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.501753 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-token\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.501787 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.501791 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzldr\" (UniqueName: \"kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-kube-api-access-zzldr\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502088 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.501827 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-certs\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502088 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.501848 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-envoy\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502088 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.501867 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-podinfo\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502088 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.501940 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-socket\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502088 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.501990 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-data\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502088 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.502029 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-credential-socket\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502088 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.502057 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9891e388-585d-4aad-8757-f0c6ea6536e8-istiod-ca-cert\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502442 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.502274 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-certs\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502442 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.502370 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-socket\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502599 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.502455 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-credential-socket\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502731 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.502711 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9891e388-585d-4aad-8757-f0c6ea6536e8-istiod-ca-cert\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.502797 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.502740 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-data\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.504306 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.504283 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-envoy\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.504623 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.504510 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-podinfo\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.509972 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.509951 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-token\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.510184 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.510166 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzldr\" (UniqueName: \"kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-kube-api-access-zzldr\") pod \"e2e-inf-b-85c487-openshift-default-54979d78b9-62njj\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.610325 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.610292 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:25.754000 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.753969 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj"] Jul 22 19:31:25.755212 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:31:25.755186 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9891e388_585d_4aad_8757_f0c6ea6536e8.slice/crio-7e794f60b0bcb1e7ddbac6b92e42f87d60112ea516c18fe5d834984a2f9e46db WatchSource:0}: Error finding container 7e794f60b0bcb1e7ddbac6b92e42f87d60112ea516c18fe5d834984a2f9e46db: Status 404 returned error can't find the container with id 7e794f60b0bcb1e7ddbac6b92e42f87d60112ea516c18fe5d834984a2f9e46db Jul 22 19:31:25.757509 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.757474 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:25.757619 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.757565 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:25.757619 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:25.757594 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:26.327800 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:26.327761 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" event={"ID":"9891e388-585d-4aad-8757-f0c6ea6536e8","Type":"ContainerStarted","Data":"6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599"} Jul 22 19:31:26.328221 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:26.327812 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" event={"ID":"9891e388-585d-4aad-8757-f0c6ea6536e8","Type":"ContainerStarted","Data":"7e794f60b0bcb1e7ddbac6b92e42f87d60112ea516c18fe5d834984a2f9e46db"} Jul 22 19:31:26.349793 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:26.349749 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" podStartSLOduration=1.349733879 podStartE2EDuration="1.349733879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:31:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:31:26.347748069 +0000 UTC m=+2420.010322414" watchObservedRunningTime="2026-07-22 19:31:26.349733879 +0000 UTC m=+2420.012308199" Jul 22 19:31:26.610825 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:26.610781 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:26.615838 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:26.615809 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:27.333228 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:27.333197 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:27.334136 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:27.334120 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:36.135560 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:36.135499 2584 patch_prober.go:28] interesting pod/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:58932->10.134.0.52:15021: read: connection reset by peer" start-of-body= Jul 22 19:31:36.136031 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:36.135618 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:58932->10.134.0.52:15021: read: connection reset by peer" Jul 22 19:31:36.389488 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:36.389390 2584 generic.go:356] "Generic (PLEG): container finished" podID="71b15220-e472-4d30-9157-465c8073f513" containerID="fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8" exitCode=0 Jul 22 19:31:36.389693 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:36.389498 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" event={"ID":"71b15220-e472-4d30-9157-465c8073f513","Type":"ContainerDied","Data":"fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8"} Jul 22 19:31:36.390258 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:36.390229 2584 scope.go:117] "RemoveContainer" containerID="fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8" Jul 22 19:31:36.391780 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:36.391738 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:36.391895 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:36.391835 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:36.391895 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:36.391883 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:37.400434 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:37.400398 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" event={"ID":"71b15220-e472-4d30-9157-465c8073f513","Type":"ContainerStarted","Data":"df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08"} Jul 22 19:31:37.932738 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:37.932685 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:37.933054 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:37.932927 2584 patch_prober.go:28] interesting pod/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 22 19:31:37.933054 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:37.932993 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 22 19:31:38.932540 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:38.932471 2584 patch_prober.go:28] interesting pod/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 22 19:31:38.932956 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:38.932580 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 22 19:31:39.932702 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:39.932661 2584 patch_prober.go:28] interesting pod/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 22 19:31:39.933096 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:39.932734 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 22 19:31:40.932243 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:40.932200 2584 patch_prober.go:28] interesting pod/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 22 19:31:40.932402 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:40.932273 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 22 19:31:41.937199 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:41.937161 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:41.937624 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:41.937509 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:41.938269 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:41.938249 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:31:42.334816 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:42.334782 2584 patch_prober.go:28] interesting pod/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 22 19:31:42.334966 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:42.334856 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 22 19:31:42.432210 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:42.432175 2584 generic.go:356] "Generic (PLEG): container finished" podID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerID="6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599" exitCode=0 Jul 22 19:31:42.432372 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:42.432250 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" event={"ID":"9891e388-585d-4aad-8757-f0c6ea6536e8","Type":"ContainerDied","Data":"6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599"} Jul 22 19:31:42.432751 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:42.432738 2584 scope.go:117] "RemoveContainer" containerID="6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599" Jul 22 19:31:42.433685 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:42.433659 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:42.433986 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:42.433730 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:42.433986 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:42.433765 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:31:43.438933 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:43.438900 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" event={"ID":"9891e388-585d-4aad-8757-f0c6ea6536e8","Type":"ContainerStarted","Data":"020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200"} Jul 22 19:31:43.611094 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:43.611049 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:43.611358 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:43.611326 2584 patch_prober.go:28] interesting pod/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 22 19:31:43.611498 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:43.611385 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 22 19:31:44.611128 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:44.611084 2584 patch_prober.go:28] interesting pod/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 22 19:31:44.611644 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:44.611145 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 22 19:31:45.611555 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:45.611494 2584 patch_prober.go:28] interesting pod/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 22 19:31:45.611929 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:45.611574 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 22 19:31:46.611694 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:46.611654 2584 patch_prober.go:28] interesting pod/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 22 19:31:46.612171 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:46.611716 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 22 19:31:47.614544 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:47.614490 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:47.614896 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:47.614863 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:31:47.615490 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:31:47.615473 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:32:48.180013 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:48.179971 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt"] Jul 22 19:32:48.182981 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:48.182961 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt" Jul 22 19:32:48.185457 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:48.185426 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\"" Jul 22 19:32:48.189241 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:48.189214 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt"] Jul 22 19:32:48.251851 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:48.251816 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f369a292-5a50-44f1-b274-631ae85554b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt\" (UID: \"f369a292-5a50-44f1-b274-631ae85554b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt" Jul 22 19:32:48.352843 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:48.352804 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f369a292-5a50-44f1-b274-631ae85554b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt\" (UID: \"f369a292-5a50-44f1-b274-631ae85554b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt" Jul 22 19:32:48.353459 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:48.353426 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f369a292-5a50-44f1-b274-631ae85554b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt\" (UID: \"f369a292-5a50-44f1-b274-631ae85554b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt" Jul 22 19:32:48.496889 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:48.496801 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt" Jul 22 19:32:48.635239 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:48.635214 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt"] Jul 22 19:32:48.637453 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:32:48.637429 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf369a292_5a50_44f1_b274_631ae85554b4.slice/crio-beec84d7218009b753691c4b274eefad3c45512f7b0d868db0ce0ff53002c191 WatchSource:0}: Error finding container beec84d7218009b753691c4b274eefad3c45512f7b0d868db0ce0ff53002c191: Status 404 returned error can't find the container with id beec84d7218009b753691c4b274eefad3c45512f7b0d868db0ce0ff53002c191 Jul 22 19:32:48.777018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:48.776925 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt" event={"ID":"f369a292-5a50-44f1-b274-631ae85554b4","Type":"ContainerStarted","Data":"beec84d7218009b753691c4b274eefad3c45512f7b0d868db0ce0ff53002c191"} Jul 22 19:32:49.784483 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:49.784388 2584 generic.go:356] "Generic (PLEG): container finished" podID="f369a292-5a50-44f1-b274-631ae85554b4" containerID="ece7d15b1a54617c3311a763b0a1bec5f8514de524325cab7f74a8cb3f420bbe" exitCode=0 Jul 22 19:32:49.784483 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:49.784446 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt" event={"ID":"f369a292-5a50-44f1-b274-631ae85554b4","Type":"ContainerDied","Data":"ece7d15b1a54617c3311a763b0a1bec5f8514de524325cab7f74a8cb3f420bbe"} Jul 22 19:32:50.919235 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:50.919207 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt" Jul 22 19:32:50.978115 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:50.978082 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f369a292-5a50-44f1-b274-631ae85554b4-maas-api-service-ca\") pod \"f369a292-5a50-44f1-b274-631ae85554b4\" (UID: \"f369a292-5a50-44f1-b274-631ae85554b4\") " Jul 22 19:32:50.978474 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:50.978446 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f369a292-5a50-44f1-b274-631ae85554b4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f369a292-5a50-44f1-b274-631ae85554b4" (UID: "f369a292-5a50-44f1-b274-631ae85554b4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:32:51.079017 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:51.078993 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f369a292-5a50-44f1-b274-631ae85554b4-maas-api-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:32:51.796132 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:51.796097 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt" Jul 22 19:32:51.796307 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:51.796144 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt" event={"ID":"f369a292-5a50-44f1-b274-631ae85554b4","Type":"ContainerDied","Data":"beec84d7218009b753691c4b274eefad3c45512f7b0d868db0ce0ff53002c191"} Jul 22 19:32:51.796307 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:32:51.796191 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="beec84d7218009b753691c4b274eefad3c45512f7b0d868db0ce0ff53002c191" Jul 22 19:33:03.258780 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:03.258697 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt"] Jul 22 19:33:03.260823 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:03.260798 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-9c16c7-llkgt"] Jul 22 19:33:04.906212 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:04.906176 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f369a292-5a50-44f1-b274-631ae85554b4" path="/var/lib/kubelet/pods/f369a292-5a50-44f1-b274-631ae85554b4/volumes" Jul 22 19:33:09.668678 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:09.668644 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8"] Jul 22 19:33:09.669136 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:09.668902 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" containerID="cri-o://df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08" gracePeriod=30 Jul 22 19:33:10.049297 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.049208 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s"] Jul 22 19:33:10.049647 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.049634 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f369a292-5a50-44f1-b274-631ae85554b4" containerName="revoke-keys" Jul 22 19:33:10.049697 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.049649 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f369a292-5a50-44f1-b274-631ae85554b4" containerName="revoke-keys" Jul 22 19:33:10.049741 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.049718 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f369a292-5a50-44f1-b274-631ae85554b4" containerName="revoke-keys" Jul 22 19:33:10.051810 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.051792 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s" Jul 22 19:33:10.054189 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.054170 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\"" Jul 22 19:33:10.058488 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.058465 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s"] Jul 22 19:33:10.155385 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.155349 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fbfed6-b49a-423f-89e2-9aec904ec09f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-85c487-5s57s\" (UID: \"c3fbfed6-b49a-423f-89e2-9aec904ec09f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s" Jul 22 19:33:10.255864 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.255829 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fbfed6-b49a-423f-89e2-9aec904ec09f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-85c487-5s57s\" (UID: \"c3fbfed6-b49a-423f-89e2-9aec904ec09f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s" Jul 22 19:33:10.256472 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.256446 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fbfed6-b49a-423f-89e2-9aec904ec09f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-85c487-5s57s\" (UID: \"c3fbfed6-b49a-423f-89e2-9aec904ec09f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s" Jul 22 19:33:10.364772 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.364743 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s" Jul 22 19:33:10.713574 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.713537 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s"] Jul 22 19:33:10.715176 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:33:10.715149 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3fbfed6_b49a_423f_89e2_9aec904ec09f.slice/crio-6c9d9933736c161b6408e82c782129d26a5fbba1a1f69ab2cd920422a898ff30 WatchSource:0}: Error finding container 6c9d9933736c161b6408e82c782129d26a5fbba1a1f69ab2cd920422a898ff30: Status 404 returned error can't find the container with id 6c9d9933736c161b6408e82c782129d26a5fbba1a1f69ab2cd920422a898ff30 Jul 22 19:33:10.906248 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:10.906213 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s" event={"ID":"c3fbfed6-b49a-423f-89e2-9aec904ec09f","Type":"ContainerStarted","Data":"6c9d9933736c161b6408e82c782129d26a5fbba1a1f69ab2cd920422a898ff30"} Jul 22 19:33:11.908961 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:11.908927 2584 generic.go:356] "Generic (PLEG): container finished" podID="c3fbfed6-b49a-423f-89e2-9aec904ec09f" containerID="9e40f85a181a391e462e4904afd07119a5b7a95528d908ebacd9f8c25b527ec9" exitCode=0 Jul 22 19:33:11.909357 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:11.908967 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s" event={"ID":"c3fbfed6-b49a-423f-89e2-9aec904ec09f","Type":"ContainerDied","Data":"9e40f85a181a391e462e4904afd07119a5b7a95528d908ebacd9f8c25b527ec9"} Jul 22 19:33:11.939671 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:11.939644 2584 patch_prober.go:28] interesting pod/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:33:11.939830 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:11.939699 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:33:13.053097 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:13.053072 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s" Jul 22 19:33:13.185867 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:13.185787 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fbfed6-b49a-423f-89e2-9aec904ec09f-maas-api-service-ca\") pod \"c3fbfed6-b49a-423f-89e2-9aec904ec09f\" (UID: \"c3fbfed6-b49a-423f-89e2-9aec904ec09f\") " Jul 22 19:33:13.186179 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:13.186159 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3fbfed6-b49a-423f-89e2-9aec904ec09f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c3fbfed6-b49a-423f-89e2-9aec904ec09f" (UID: "c3fbfed6-b49a-423f-89e2-9aec904ec09f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:33:13.286606 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:13.286582 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3fbfed6-b49a-423f-89e2-9aec904ec09f-maas-api-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:13.919497 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:13.919464 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s" Jul 22 19:33:13.919690 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:13.919465 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s" event={"ID":"c3fbfed6-b49a-423f-89e2-9aec904ec09f","Type":"ContainerDied","Data":"6c9d9933736c161b6408e82c782129d26a5fbba1a1f69ab2cd920422a898ff30"} Jul 22 19:33:13.919690 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:13.919581 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c9d9933736c161b6408e82c782129d26a5fbba1a1f69ab2cd920422a898ff30" Jul 22 19:33:14.924457 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:14.924428 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:33:14.925907 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:14.925884 2584 generic.go:356] "Generic (PLEG): container finished" podID="71b15220-e472-4d30-9157-465c8073f513" containerID="df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08" exitCode=0 Jul 22 19:33:14.926009 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:14.925929 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" event={"ID":"71b15220-e472-4d30-9157-465c8073f513","Type":"ContainerDied","Data":"df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08"} Jul 22 19:33:14.926009 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:14.925974 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" event={"ID":"71b15220-e472-4d30-9157-465c8073f513","Type":"ContainerDied","Data":"35655ffafb6885d5b07809b50ddbcde2b6fa496b5bbf7e480a8abe7a28e2e02c"} Jul 22 19:33:14.926009 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:14.925997 2584 scope.go:117] "RemoveContainer" containerID="df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08" Jul 22 19:33:14.935669 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:14.935641 2584 scope.go:117] "RemoveContainer" containerID="fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8" Jul 22 19:33:14.946225 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:14.946194 2584 scope.go:117] "RemoveContainer" containerID="df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08" Jul 22 19:33:14.946599 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:33:14.946574 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08\": container with ID starting with df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08 not found: ID does not exist" containerID="df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08" Jul 22 19:33:14.946693 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:14.946610 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08"} err="failed to get container status \"df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08\": rpc error: code = NotFound desc = could not find container \"df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08\": container with ID starting with df1d1a6f60113eb25f36e6036bc8419922dd6740b0f75f25b0deeaf3ba2b8c08 not found: ID does not exist" Jul 22 19:33:14.946693 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:14.946636 2584 scope.go:117] "RemoveContainer" containerID="fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8" Jul 22 19:33:14.947069 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:33:14.947038 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8\": container with ID starting with fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8 not found: ID does not exist" containerID="fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8" Jul 22 19:33:14.947156 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:14.947085 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8"} err="failed to get container status \"fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8\": rpc error: code = NotFound desc = could not find container \"fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8\": container with ID starting with fad494bcbe4c1b5faa7256586af6238d80eabce668c3756a6414ea354670b4b8 not found: ID does not exist" Jul 22 19:33:15.101875 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.101841 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-data\") pod \"71b15220-e472-4d30-9157-465c8073f513\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " Jul 22 19:33:15.102058 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.101884 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-istio-token\") pod \"71b15220-e472-4d30-9157-465c8073f513\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " Jul 22 19:33:15.102058 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.101930 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b15220-e472-4d30-9157-465c8073f513-istiod-ca-cert\") pod \"71b15220-e472-4d30-9157-465c8073f513\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " Jul 22 19:33:15.102058 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.101983 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-certs\") pod \"71b15220-e472-4d30-9157-465c8073f513\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " Jul 22 19:33:15.102058 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102008 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-credential-socket\") pod \"71b15220-e472-4d30-9157-465c8073f513\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " Jul 22 19:33:15.102058 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102049 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-socket\") pod \"71b15220-e472-4d30-9157-465c8073f513\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " Jul 22 19:33:15.102366 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102075 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-envoy\") pod \"71b15220-e472-4d30-9157-465c8073f513\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " Jul 22 19:33:15.102366 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102163 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbqxh\" (UniqueName: \"kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-kube-api-access-cbqxh\") pod \"71b15220-e472-4d30-9157-465c8073f513\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " Jul 22 19:33:15.102366 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102207 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b15220-e472-4d30-9157-465c8073f513-istio-podinfo\") pod \"71b15220-e472-4d30-9157-465c8073f513\" (UID: \"71b15220-e472-4d30-9157-465c8073f513\") " Jul 22 19:33:15.102366 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102347 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71b15220-e472-4d30-9157-465c8073f513" (UID: "71b15220-e472-4d30-9157-465c8073f513"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:33:15.102620 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102359 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71b15220-e472-4d30-9157-465c8073f513" (UID: "71b15220-e472-4d30-9157-465c8073f513"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:33:15.102620 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102348 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71b15220-e472-4d30-9157-465c8073f513" (UID: "71b15220-e472-4d30-9157-465c8073f513"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:33:15.102620 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102406 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71b15220-e472-4d30-9157-465c8073f513-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71b15220-e472-4d30-9157-465c8073f513" (UID: "71b15220-e472-4d30-9157-465c8073f513"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:33:15.102620 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102555 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71b15220-e472-4d30-9157-465c8073f513-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:15.102620 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102575 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:15.102620 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102587 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:15.102620 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102600 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:15.102932 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.102830 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-data" (OuterVolumeSpecName: "istio-data") pod "71b15220-e472-4d30-9157-465c8073f513" (UID: "71b15220-e472-4d30-9157-465c8073f513"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:33:15.104180 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.104151 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-istio-token" (OuterVolumeSpecName: "istio-token") pod "71b15220-e472-4d30-9157-465c8073f513" (UID: "71b15220-e472-4d30-9157-465c8073f513"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:33:15.104369 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.104350 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71b15220-e472-4d30-9157-465c8073f513-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71b15220-e472-4d30-9157-465c8073f513" (UID: "71b15220-e472-4d30-9157-465c8073f513"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:33:15.104460 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.104444 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71b15220-e472-4d30-9157-465c8073f513" (UID: "71b15220-e472-4d30-9157-465c8073f513"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:33:15.104581 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.104564 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-kube-api-access-cbqxh" (OuterVolumeSpecName: "kube-api-access-cbqxh") pod "71b15220-e472-4d30-9157-465c8073f513" (UID: "71b15220-e472-4d30-9157-465c8073f513"). InnerVolumeSpecName "kube-api-access-cbqxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:33:15.203938 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.203872 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbqxh\" (UniqueName: \"kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-kube-api-access-cbqxh\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:15.203938 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.203899 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71b15220-e472-4d30-9157-465c8073f513-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:15.203938 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.203912 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:15.203938 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.203924 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71b15220-e472-4d30-9157-465c8073f513-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:15.203938 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.203936 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71b15220-e472-4d30-9157-465c8073f513-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:15.932701 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.932672 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8" Jul 22 19:33:15.955885 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.955858 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8"] Jul 22 19:33:15.959559 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:15.959536 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-9c16c7-openshift-default-64997b7c4c-rhzz8"] Jul 22 19:33:16.907664 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:16.907632 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71b15220-e472-4d30-9157-465c8073f513" path="/var/lib/kubelet/pods/71b15220-e472-4d30-9157-465c8073f513/volumes" Jul 22 19:33:25.106236 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:25.106196 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s"] Jul 22 19:33:25.108991 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:25.108965 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-85c487-5s57s"] Jul 22 19:33:26.912166 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:26.912128 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3fbfed6-b49a-423f-89e2-9aec904ec09f" path="/var/lib/kubelet/pods/c3fbfed6-b49a-423f-89e2-9aec904ec09f/volumes" Jul 22 19:33:31.479402 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:31.479368 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj"] Jul 22 19:33:31.479975 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:31.479648 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" containerID="cri-o://020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200" gracePeriod=30 Jul 22 19:33:32.346908 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.346749 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb"] Jul 22 19:33:32.347429 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.347411 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3fbfed6-b49a-423f-89e2-9aec904ec09f" containerName="revoke-keys" Jul 22 19:33:32.347558 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.347432 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3fbfed6-b49a-423f-89e2-9aec904ec09f" containerName="revoke-keys" Jul 22 19:33:32.347558 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.347449 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" Jul 22 19:33:32.347558 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.347458 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" Jul 22 19:33:32.347734 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.347572 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" Jul 22 19:33:32.347734 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.347585 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3fbfed6-b49a-423f-89e2-9aec904ec09f" containerName="revoke-keys" Jul 22 19:33:32.350961 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.350940 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.353583 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.353564 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-802be2-openshift-default-dockercfg-dxxph\"" Jul 22 19:33:32.361315 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.361288 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb"] Jul 22 19:33:32.447654 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.447619 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-envoy\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.447654 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.447655 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnpk9\" (UniqueName: \"kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-kube-api-access-qnpk9\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.447884 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.447687 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-podinfo\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.447884 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.447744 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-certs\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.447884 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.447793 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-data\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.447884 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.447878 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-socket\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.448021 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.447895 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-credential-socket\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.448021 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.447921 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istiod-ca-cert\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.448095 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.448010 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-token\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.549038 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549007 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-envoy\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.549038 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549042 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnpk9\" (UniqueName: \"kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-kube-api-access-qnpk9\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.549638 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549063 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-podinfo\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.549638 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549094 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-certs\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.549638 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549156 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-data\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.549638 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549205 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-socket\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.549638 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549227 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-credential-socket\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.549638 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549254 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istiod-ca-cert\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.549638 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549284 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-token\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.549638 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549564 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-certs\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.550248 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549673 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-socket\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.550248 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549708 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-data\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.550248 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.549897 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-credential-socket\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.550248 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.550196 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istiod-ca-cert\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.551450 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.551424 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-envoy\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.551768 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.551750 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-podinfo\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.557203 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.557176 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-token\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.557203 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.557185 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnpk9\" (UniqueName: \"kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-kube-api-access-qnpk9\") pod \"e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.617158 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.617085 2584 patch_prober.go:28] interesting pod/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:33:32.617158 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.617137 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:33:32.663066 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.663035 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:32.797958 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.797928 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb"] Jul 22 19:33:32.799090 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:33:32.799058 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6763c4_a8ac_4c7d_88c7_3a3808ae7115.slice/crio-7df091faed0653b060ff4db86ac1cd1c2c457a0c4c3d44ddbfcc73fbd4b7f676 WatchSource:0}: Error finding container 7df091faed0653b060ff4db86ac1cd1c2c457a0c4c3d44ddbfcc73fbd4b7f676: Status 404 returned error can't find the container with id 7df091faed0653b060ff4db86ac1cd1c2c457a0c4c3d44ddbfcc73fbd4b7f676 Jul 22 19:33:32.801190 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.801155 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:33:32.801276 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.801236 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:33:32.801276 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:32.801262 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:33:33.029187 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:33.029091 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" event={"ID":"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115","Type":"ContainerStarted","Data":"bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d"} Jul 22 19:33:33.029187 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:33.029146 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" event={"ID":"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115","Type":"ContainerStarted","Data":"7df091faed0653b060ff4db86ac1cd1c2c457a0c4c3d44ddbfcc73fbd4b7f676"} Jul 22 19:33:33.051979 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:33.051917 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" podStartSLOduration=1.051896589 podStartE2EDuration="1.051896589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:33:33.048493167 +0000 UTC m=+2546.711067506" watchObservedRunningTime="2026-07-22 19:33:33.051896589 +0000 UTC m=+2546.714470931" Jul 22 19:33:33.663831 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:33.663790 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:33.669046 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:33.669019 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:34.034079 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:34.033991 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:34.034892 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:34.034871 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:36.722168 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.722145 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:33:36.792818 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.792748 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-data\") pod \"9891e388-585d-4aad-8757-f0c6ea6536e8\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " Jul 22 19:33:36.792818 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.792805 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-token\") pod \"9891e388-585d-4aad-8757-f0c6ea6536e8\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " Jul 22 19:33:36.793013 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.792830 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-certs\") pod \"9891e388-585d-4aad-8757-f0c6ea6536e8\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " Jul 22 19:33:36.793013 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.792860 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-podinfo\") pod \"9891e388-585d-4aad-8757-f0c6ea6536e8\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " Jul 22 19:33:36.793013 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.792888 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9891e388-585d-4aad-8757-f0c6ea6536e8-istiod-ca-cert\") pod \"9891e388-585d-4aad-8757-f0c6ea6536e8\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " Jul 22 19:33:36.793013 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.792930 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-credential-socket\") pod \"9891e388-585d-4aad-8757-f0c6ea6536e8\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " Jul 22 19:33:36.793202 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.793115 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-envoy\") pod \"9891e388-585d-4aad-8757-f0c6ea6536e8\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " Jul 22 19:33:36.793202 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.793175 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzldr\" (UniqueName: \"kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-kube-api-access-zzldr\") pod \"9891e388-585d-4aad-8757-f0c6ea6536e8\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " Jul 22 19:33:36.793315 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.793221 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-socket\") pod \"9891e388-585d-4aad-8757-f0c6ea6536e8\" (UID: \"9891e388-585d-4aad-8757-f0c6ea6536e8\") " Jul 22 19:33:36.793975 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.793219 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9891e388-585d-4aad-8757-f0c6ea6536e8" (UID: "9891e388-585d-4aad-8757-f0c6ea6536e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:33:36.793975 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.793655 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9891e388-585d-4aad-8757-f0c6ea6536e8" (UID: "9891e388-585d-4aad-8757-f0c6ea6536e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:33:36.793975 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.793749 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "9891e388-585d-4aad-8757-f0c6ea6536e8" (UID: "9891e388-585d-4aad-8757-f0c6ea6536e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:33:36.793975 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.793832 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9891e388-585d-4aad-8757-f0c6ea6536e8" (UID: "9891e388-585d-4aad-8757-f0c6ea6536e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:33:36.793975 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.793966 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9891e388-585d-4aad-8757-f0c6ea6536e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9891e388-585d-4aad-8757-f0c6ea6536e8" (UID: "9891e388-585d-4aad-8757-f0c6ea6536e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:33:36.795287 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.795265 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "9891e388-585d-4aad-8757-f0c6ea6536e8" (UID: "9891e388-585d-4aad-8757-f0c6ea6536e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:33:36.795420 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.795401 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9891e388-585d-4aad-8757-f0c6ea6536e8" (UID: "9891e388-585d-4aad-8757-f0c6ea6536e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:33:36.795540 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.795489 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9891e388-585d-4aad-8757-f0c6ea6536e8" (UID: "9891e388-585d-4aad-8757-f0c6ea6536e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:33:36.795714 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.795689 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-kube-api-access-zzldr" (OuterVolumeSpecName: "kube-api-access-zzldr") pod "9891e388-585d-4aad-8757-f0c6ea6536e8" (UID: "9891e388-585d-4aad-8757-f0c6ea6536e8"). InnerVolumeSpecName "kube-api-access-zzldr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:33:36.894878 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.894842 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:36.894878 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.894870 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:36.894878 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.894881 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzldr\" (UniqueName: \"kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-kube-api-access-zzldr\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:36.895135 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.894890 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:36.895135 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.894899 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:36.895135 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.894907 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:36.895135 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.894914 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9891e388-585d-4aad-8757-f0c6ea6536e8-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:36.895135 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.894922 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9891e388-585d-4aad-8757-f0c6ea6536e8-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:36.895135 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:36.894928 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9891e388-585d-4aad-8757-f0c6ea6536e8-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:37.051112 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.051020 2584 generic.go:356] "Generic (PLEG): container finished" podID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerID="020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200" exitCode=0 Jul 22 19:33:37.051112 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.051100 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" Jul 22 19:33:37.051282 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.051117 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" event={"ID":"9891e388-585d-4aad-8757-f0c6ea6536e8","Type":"ContainerDied","Data":"020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200"} Jul 22 19:33:37.051282 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.051220 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj" event={"ID":"9891e388-585d-4aad-8757-f0c6ea6536e8","Type":"ContainerDied","Data":"7e794f60b0bcb1e7ddbac6b92e42f87d60112ea516c18fe5d834984a2f9e46db"} Jul 22 19:33:37.051282 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.051253 2584 scope.go:117] "RemoveContainer" containerID="020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200" Jul 22 19:33:37.060378 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.060361 2584 scope.go:117] "RemoveContainer" containerID="6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599" Jul 22 19:33:37.068977 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.068960 2584 scope.go:117] "RemoveContainer" containerID="020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200" Jul 22 19:33:37.069265 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:33:37.069238 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200\": container with ID starting with 020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200 not found: ID does not exist" containerID="020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200" Jul 22 19:33:37.069366 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.069270 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200"} err="failed to get container status \"020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200\": rpc error: code = NotFound desc = could not find container \"020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200\": container with ID starting with 020a007d58749c33c910e04f5180398eab8f64db06f96d8860b745f9ef65c200 not found: ID does not exist" Jul 22 19:33:37.069366 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.069287 2584 scope.go:117] "RemoveContainer" containerID="6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599" Jul 22 19:33:37.069558 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:33:37.069539 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599\": container with ID starting with 6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599 not found: ID does not exist" containerID="6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599" Jul 22 19:33:37.069612 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.069564 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599"} err="failed to get container status \"6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599\": rpc error: code = NotFound desc = could not find container \"6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599\": container with ID starting with 6dae46880c0aa562dd4422c483af61eedf7435a44069cb226b31dc6ccd57b599 not found: ID does not exist" Jul 22 19:33:37.069816 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.069798 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj"] Jul 22 19:33:37.073160 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:37.073136 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-85c487-openshift-default-54979d78b9-62njj"] Jul 22 19:33:38.906943 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:38.906908 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" path="/var/lib/kubelet/pods/9891e388-585d-4aad-8757-f0c6ea6536e8/volumes" Jul 22 19:33:40.252911 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.252877 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4"] Jul 22 19:33:40.253404 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.253390 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" Jul 22 19:33:40.253451 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.253407 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" Jul 22 19:33:40.253451 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.253423 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" Jul 22 19:33:40.253451 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.253432 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" Jul 22 19:33:40.253591 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.253555 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="71b15220-e472-4d30-9157-465c8073f513" containerName="istio-proxy" Jul 22 19:33:40.253591 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.253568 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" Jul 22 19:33:40.253591 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.253578 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" Jul 22 19:33:40.257554 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.257533 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.260796 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.260778 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-61193d-openshift-default-dockercfg-zl5nc\"" Jul 22 19:33:40.267095 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.267060 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4"] Jul 22 19:33:40.328347 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.328315 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-socket\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.328483 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.328352 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-credential-socket\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.328483 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.328410 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-certs\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.328483 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.328447 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-token\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.328688 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.328496 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-podinfo\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.328688 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.328532 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-data\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.328688 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.328579 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk258\" (UniqueName: \"kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-kube-api-access-lk258\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.328688 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.328638 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-envoy\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.328688 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.328673 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istiod-ca-cert\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.429938 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.429907 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-envoy\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430105 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.429947 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istiod-ca-cert\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430105 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430083 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-socket\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430237 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430118 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-credential-socket\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430237 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430164 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-certs\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430237 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430202 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-token\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430383 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430281 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-podinfo\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430383 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430319 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-data\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430383 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430351 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk258\" (UniqueName: \"kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-kube-api-access-lk258\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430565 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430441 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-socket\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430565 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430508 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-credential-socket\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430679 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430585 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-certs\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430679 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430648 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istiod-ca-cert\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.430961 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.430789 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-data\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.432427 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.432402 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-envoy\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.432954 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.432933 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-podinfo\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.438778 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.438756 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-token\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.439126 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.439105 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk258\" (UniqueName: \"kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-kube-api-access-lk258\") pod \"e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.570486 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.570402 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:40.717772 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.717725 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4"] Jul 22 19:33:40.718481 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:33:40.718442 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa3c73b_db31_4204_99d3_f841e2aeb7b3.slice/crio-d7785f81c903e97a0d443ca1e0da037d16466963d82be3ff2b09dfc7c2bb2525 WatchSource:0}: Error finding container d7785f81c903e97a0d443ca1e0da037d16466963d82be3ff2b09dfc7c2bb2525: Status 404 returned error can't find the container with id d7785f81c903e97a0d443ca1e0da037d16466963d82be3ff2b09dfc7c2bb2525 Jul 22 19:33:40.720715 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.720686 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:33:40.720794 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.720750 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:33:40.720794 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:40.720777 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:33:41.074332 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:41.074295 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" event={"ID":"9fa3c73b-db31-4204-99d3-f841e2aeb7b3","Type":"ContainerStarted","Data":"eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509"} Jul 22 19:33:41.074748 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:41.074358 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" event={"ID":"9fa3c73b-db31-4204-99d3-f841e2aeb7b3","Type":"ContainerStarted","Data":"d7785f81c903e97a0d443ca1e0da037d16466963d82be3ff2b09dfc7c2bb2525"} Jul 22 19:33:41.093080 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:41.093025 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" podStartSLOduration=1.093007007 podStartE2EDuration="1.093007007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:33:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:33:41.092651161 +0000 UTC m=+2554.755225502" watchObservedRunningTime="2026-07-22 19:33:41.093007007 +0000 UTC m=+2554.755581347" Jul 22 19:33:41.571593 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:41.571561 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:41.577081 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:41.577057 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:42.087964 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:42.087621 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:42.088764 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:42.088741 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:33:50.076672 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:50.076623 2584 patch_prober.go:28] interesting pod/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 19:33:50.077050 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:50.076705 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 19:33:52.148684 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:52.148646 2584 generic.go:356] "Generic (PLEG): container finished" podID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerID="bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d" exitCode=0 Jul 22 19:33:52.149118 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:52.148727 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" event={"ID":"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115","Type":"ContainerDied","Data":"bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d"} Jul 22 19:33:52.149277 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:52.149259 2584 scope.go:117] "RemoveContainer" containerID="bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d" Jul 22 19:33:52.150530 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:52.150483 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:33:52.150611 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:52.150590 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:33:52.150657 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:52.150640 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:33:53.156078 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:53.156044 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" event={"ID":"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115","Type":"ContainerStarted","Data":"4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b"} Jul 22 19:33:53.663823 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:53.663779 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:53.664094 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:53.664068 2584 patch_prober.go:28] interesting pod/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 22 19:33:53.664188 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:53.664129 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 22 19:33:54.664321 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:54.664281 2584 patch_prober.go:28] interesting pod/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 22 19:33:54.664734 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:54.664348 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 22 19:33:55.576302 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.576269 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j"] Jul 22 19:33:55.576732 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.576713 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" Jul 22 19:33:55.576834 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.576733 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9891e388-585d-4aad-8757-f0c6ea6536e8" containerName="istio-proxy" Jul 22 19:33:55.578755 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.578735 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j" Jul 22 19:33:55.581196 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.581174 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\"" Jul 22 19:33:55.587881 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.587850 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j"] Jul 22 19:33:55.592651 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.592564 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbbf1de-7fee-4855-b50c-84ab5b556831-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-802be2-l2v5j\" (UID: \"bcbbf1de-7fee-4855-b50c-84ab5b556831\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j" Jul 22 19:33:55.663984 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.663951 2584 patch_prober.go:28] interesting pod/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 22 19:33:55.664168 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.664099 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 22 19:33:55.712179 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.695665 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbbf1de-7fee-4855-b50c-84ab5b556831-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-802be2-l2v5j\" (UID: \"bcbbf1de-7fee-4855-b50c-84ab5b556831\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j" Jul 22 19:33:55.712179 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.696587 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbbf1de-7fee-4855-b50c-84ab5b556831-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-802be2-l2v5j\" (UID: \"bcbbf1de-7fee-4855-b50c-84ab5b556831\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j" Jul 22 19:33:55.901935 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:55.901894 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j" Jul 22 19:33:56.130544 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:56.130158 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j"] Jul 22 19:33:56.174810 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:56.174783 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j" event={"ID":"bcbbf1de-7fee-4855-b50c-84ab5b556831","Type":"ContainerStarted","Data":"4e29f4d82cb1d49a6d132d4069c483afb95db1db793e332a59e6a58fa4ceda45"} Jul 22 19:33:56.663800 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:56.663761 2584 patch_prober.go:28] interesting pod/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 22 19:33:56.664003 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:56.663828 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 22 19:33:57.183838 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:57.183807 2584 generic.go:356] "Generic (PLEG): container finished" podID="bcbbf1de-7fee-4855-b50c-84ab5b556831" containerID="bee59c265ce974e0219547209194e95d81b56aff287bcdaa8ffe003cd658c0b9" exitCode=0 Jul 22 19:33:57.184243 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:57.183891 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j" event={"ID":"bcbbf1de-7fee-4855-b50c-84ab5b556831","Type":"ContainerDied","Data":"bee59c265ce974e0219547209194e95d81b56aff287bcdaa8ffe003cd658c0b9"} Jul 22 19:33:57.668774 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:57.668741 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:57.669235 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:57.669209 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:57.669920 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:57.669900 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:33:58.129689 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:58.129647 2584 patch_prober.go:28] interesting pod/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 19:33:58.129882 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:58.129711 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 19:33:58.328496 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:58.328472 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j" Jul 22 19:33:58.449395 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:58.449304 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbbf1de-7fee-4855-b50c-84ab5b556831-maas-api-service-ca\") pod \"bcbbf1de-7fee-4855-b50c-84ab5b556831\" (UID: \"bcbbf1de-7fee-4855-b50c-84ab5b556831\") " Jul 22 19:33:58.449691 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:58.449669 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcbbf1de-7fee-4855-b50c-84ab5b556831-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bcbbf1de-7fee-4855-b50c-84ab5b556831" (UID: "bcbbf1de-7fee-4855-b50c-84ab5b556831"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:33:58.550810 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:58.550773 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbbf1de-7fee-4855-b50c-84ab5b556831-maas-api-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:33:59.197360 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:59.197271 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j" Jul 22 19:33:59.197360 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:59.197311 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j" event={"ID":"bcbbf1de-7fee-4855-b50c-84ab5b556831","Type":"ContainerDied","Data":"4e29f4d82cb1d49a6d132d4069c483afb95db1db793e332a59e6a58fa4ceda45"} Jul 22 19:33:59.197360 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:33:59.197355 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e29f4d82cb1d49a6d132d4069c483afb95db1db793e332a59e6a58fa4ceda45" Jul 22 19:34:00.204731 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:00.204693 2584 generic.go:356] "Generic (PLEG): container finished" podID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerID="eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509" exitCode=0 Jul 22 19:34:00.205213 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:00.204765 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" event={"ID":"9fa3c73b-db31-4204-99d3-f841e2aeb7b3","Type":"ContainerDied","Data":"eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509"} Jul 22 19:34:00.205213 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:00.205199 2584 scope.go:117] "RemoveContainer" containerID="eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509" Jul 22 19:34:00.206196 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:00.206172 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:34:00.206249 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:00.206234 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:34:00.206283 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:00.206270 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:34:01.212670 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:01.212633 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" event={"ID":"9fa3c73b-db31-4204-99d3-f841e2aeb7b3","Type":"ContainerStarted","Data":"9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e"} Jul 22 19:34:01.571190 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:01.571099 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:34:01.571339 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:01.571255 2584 patch_prober.go:28] interesting pod/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 22 19:34:01.571339 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:01.571305 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 22 19:34:02.571611 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:02.571574 2584 patch_prober.go:28] interesting pod/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 22 19:34:02.572074 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:02.571633 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 22 19:34:03.571574 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:03.571504 2584 patch_prober.go:28] interesting pod/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 22 19:34:03.571767 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:03.571596 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 22 19:34:04.570806 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:04.570767 2584 patch_prober.go:28] interesting pod/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 22 19:34:04.570989 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:04.570822 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 22 19:34:05.571566 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:05.571507 2584 patch_prober.go:28] interesting pod/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 22 19:34:05.571979 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:05.571586 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 22 19:34:06.574810 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:06.574773 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:34:06.575259 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:06.575072 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:34:06.575842 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:06.575825 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:34:15.657347 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:15.657312 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j"] Jul 22 19:34:15.660205 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:15.660176 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-802be2-l2v5j"] Jul 22 19:34:16.907691 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:16.907658 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcbbf1de-7fee-4855-b50c-84ab5b556831" path="/var/lib/kubelet/pods/bcbbf1de-7fee-4855-b50c-84ab5b556831/volumes" Jul 22 19:34:21.862586 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:21.862548 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb"] Jul 22 19:34:21.863088 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:21.862825 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" containerID="cri-o://4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b" gracePeriod=30 Jul 22 19:34:27.104606 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.104579 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:34:27.126151 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.126082 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-socket\") pod \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " Jul 22 19:34:27.126151 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.126141 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-token\") pod \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " Jul 22 19:34:27.126370 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.126171 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-podinfo\") pod \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " Jul 22 19:34:27.126370 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.126199 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnpk9\" (UniqueName: \"kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-kube-api-access-qnpk9\") pod \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " Jul 22 19:34:27.126370 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.126225 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-credential-socket\") pod \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " Jul 22 19:34:27.126370 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.126269 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-envoy\") pod \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " Jul 22 19:34:27.126370 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.126300 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-certs\") pod \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " Jul 22 19:34:27.126370 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.126326 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istiod-ca-cert\") pod \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " Jul 22 19:34:27.126370 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.126361 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-data\") pod \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\" (UID: \"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115\") " Jul 22 19:34:27.128237 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.127454 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" (UID: "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:27.128237 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.127488 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" (UID: "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:27.128237 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.128042 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" (UID: "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:34:27.128237 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.128189 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-data" (OuterVolumeSpecName: "istio-data") pod "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" (UID: "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:27.128739 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.128712 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" (UID: "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:27.129796 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.129771 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-token" (OuterVolumeSpecName: "istio-token") pod "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" (UID: "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:34:27.130389 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.130354 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" (UID: "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:34:27.130979 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.130428 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-kube-api-access-qnpk9" (OuterVolumeSpecName: "kube-api-access-qnpk9") pod "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" (UID: "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115"). InnerVolumeSpecName "kube-api-access-qnpk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:34:27.130979 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.130433 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" (UID: "1d6763c4-a8ac-4c7d-88c7-3a3808ae7115"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:27.228164 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.228123 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:27.228164 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.228156 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:27.228164 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.228165 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:27.228410 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.228195 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnpk9\" (UniqueName: \"kubernetes.io/projected/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-kube-api-access-qnpk9\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:27.228410 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.228206 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:27.228410 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.228214 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:27.228410 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.228223 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:27.228410 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.228231 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:27.228410 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.228240 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:27.356484 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.356448 2584 generic.go:356] "Generic (PLEG): container finished" podID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerID="4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b" exitCode=0 Jul 22 19:34:27.356685 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.356503 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" Jul 22 19:34:27.356685 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.356555 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" event={"ID":"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115","Type":"ContainerDied","Data":"4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b"} Jul 22 19:34:27.356685 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.356613 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb" event={"ID":"1d6763c4-a8ac-4c7d-88c7-3a3808ae7115","Type":"ContainerDied","Data":"7df091faed0653b060ff4db86ac1cd1c2c457a0c4c3d44ddbfcc73fbd4b7f676"} Jul 22 19:34:27.356685 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.356638 2584 scope.go:117] "RemoveContainer" containerID="4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b" Jul 22 19:34:27.366983 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.366965 2584 scope.go:117] "RemoveContainer" containerID="bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d" Jul 22 19:34:27.375551 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.375532 2584 scope.go:117] "RemoveContainer" containerID="4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b" Jul 22 19:34:27.375819 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:34:27.375800 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b\": container with ID starting with 4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b not found: ID does not exist" containerID="4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b" Jul 22 19:34:27.375896 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.375827 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b"} err="failed to get container status \"4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b\": rpc error: code = NotFound desc = could not find container \"4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b\": container with ID starting with 4c88a83a3c7970bf60fb68ee9f3694acf6bfac60c52eec6e8e92f03ed7bddb2b not found: ID does not exist" Jul 22 19:34:27.375896 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.375846 2584 scope.go:117] "RemoveContainer" containerID="bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d" Jul 22 19:34:27.376085 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:34:27.376061 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d\": container with ID starting with bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d not found: ID does not exist" containerID="bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d" Jul 22 19:34:27.376139 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.376098 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d"} err="failed to get container status \"bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d\": rpc error: code = NotFound desc = could not find container \"bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d\": container with ID starting with bb5be840ec68967cf8f605cccac8b9e5c8034455c7db1cc154737c78a05c9e8d not found: ID does not exist" Jul 22 19:34:27.379813 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.379758 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb"] Jul 22 19:34:27.383381 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:27.383360 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-802be2-openshift-default-8665cd8bdf-6gzwb"] Jul 22 19:34:28.906433 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:28.906394 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" path="/var/lib/kubelet/pods/1d6763c4-a8ac-4c7d-88c7-3a3808ae7115/volumes" Jul 22 19:34:48.891103 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:48.891058 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4"] Jul 22 19:34:48.891633 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:48.891349 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" containerID="cri-o://9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e" gracePeriod=30 Jul 22 19:34:49.816018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.815982 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987"] Jul 22 19:34:49.816801 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.816772 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" Jul 22 19:34:49.816801 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.816800 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" Jul 22 19:34:49.817018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.816819 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbbf1de-7fee-4855-b50c-84ab5b556831" containerName="revoke-keys" Jul 22 19:34:49.817018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.816827 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbbf1de-7fee-4855-b50c-84ab5b556831" containerName="revoke-keys" Jul 22 19:34:49.817018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.816934 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" Jul 22 19:34:49.817018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.816947 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbbf1de-7fee-4855-b50c-84ab5b556831" containerName="revoke-keys" Jul 22 19:34:49.817018 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.816956 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" Jul 22 19:34:49.818984 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.818966 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:49.821693 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.821673 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-488695-openshift-default-dockercfg-slbwc\"" Jul 22 19:34:49.829778 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.829752 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987"] Jul 22 19:34:49.944432 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.944393 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-podinfo\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:49.944839 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.944451 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcf86\" (UniqueName: \"kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-kube-api-access-rcf86\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:49.944839 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.944531 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-data\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:49.944839 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.944589 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-envoy\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:49.944839 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.944634 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-credential-socket\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:49.944839 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.944699 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-token\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:49.944839 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.944765 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-certs\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:49.944839 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.944783 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-socket\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:49.944839 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:49.944812 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istiod-ca-cert\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.045681 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.045639 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-certs\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.045681 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.045681 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-socket\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.045950 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.045860 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istiod-ca-cert\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.045950 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.045912 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-podinfo\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.045950 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.045949 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcf86\" (UniqueName: \"kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-kube-api-access-rcf86\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.046125 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.045969 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-data\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.046125 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.045985 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-envoy\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.046125 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.046026 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-credential-socket\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.046125 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.046068 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-token\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.046328 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.046230 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-certs\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.046431 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.046405 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-data\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.046500 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.046440 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-socket\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.046803 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.046781 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istiod-ca-cert\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.047008 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.046991 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-credential-socket\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.048885 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.048856 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-envoy\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.049181 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.049158 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-podinfo\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.053974 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.053955 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-token\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.054228 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.054211 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcf86\" (UniqueName: \"kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-kube-api-access-rcf86\") pod \"e2e-shared-a-488695-openshift-default-794b69b94c-dx987\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.131127 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.131098 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:50.275285 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.275257 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987"] Jul 22 19:34:50.277311 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:34:50.277283 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7abac4_2551_4a85_b51e_1d65f5b04a13.slice/crio-5d23e5b248fd9f0cc248454e2bf8e35639479b5e431d002ff060537c04684b62 WatchSource:0}: Error finding container 5d23e5b248fd9f0cc248454e2bf8e35639479b5e431d002ff060537c04684b62: Status 404 returned error can't find the container with id 5d23e5b248fd9f0cc248454e2bf8e35639479b5e431d002ff060537c04684b62 Jul 22 19:34:50.279912 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.279875 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:34:50.279984 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.279963 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:34:50.280020 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.280004 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:34:50.484629 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.484536 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" event={"ID":"8b7abac4-2551-4a85-b51e-1d65f5b04a13","Type":"ContainerStarted","Data":"2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1"} Jul 22 19:34:50.484629 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.484602 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" event={"ID":"8b7abac4-2551-4a85-b51e-1d65f5b04a13","Type":"ContainerStarted","Data":"5d23e5b248fd9f0cc248454e2bf8e35639479b5e431d002ff060537c04684b62"} Jul 22 19:34:50.504751 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:50.504704 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" podStartSLOduration=1.504688991 podStartE2EDuration="1.504688991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:34:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:34:50.502194229 +0000 UTC m=+2624.164768571" watchObservedRunningTime="2026-07-22 19:34:50.504688991 +0000 UTC m=+2624.167263330" Jul 22 19:34:51.132082 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:51.132047 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:51.136977 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:51.136952 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:51.490452 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:51.490358 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:51.491300 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:51.491278 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:34:51.577785 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:51.577730 2584 patch_prober.go:28] interesting pod/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:34:51.577948 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:51.577800 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:34:54.139890 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.139869 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:34:54.290838 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.290752 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-certs\") pod \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " Jul 22 19:34:54.290838 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.290804 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-socket\") pod \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " Jul 22 19:34:54.290838 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.290837 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-token\") pod \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " Jul 22 19:34:54.291131 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.290866 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-credential-socket\") pod \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " Jul 22 19:34:54.291131 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.290884 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lk258\" (UniqueName: \"kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-kube-api-access-lk258\") pod \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " Jul 22 19:34:54.291131 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.290912 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-envoy\") pod \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " Jul 22 19:34:54.291131 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.290964 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istiod-ca-cert\") pod \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " Jul 22 19:34:54.291131 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.291019 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-data\") pod \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " Jul 22 19:34:54.291131 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.291053 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-podinfo\") pod \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\" (UID: \"9fa3c73b-db31-4204-99d3-f841e2aeb7b3\") " Jul 22 19:34:54.291131 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.291068 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9fa3c73b-db31-4204-99d3-f841e2aeb7b3" (UID: "9fa3c73b-db31-4204-99d3-f841e2aeb7b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:54.291495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.291162 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9fa3c73b-db31-4204-99d3-f841e2aeb7b3" (UID: "9fa3c73b-db31-4204-99d3-f841e2aeb7b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:54.291495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.291184 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9fa3c73b-db31-4204-99d3-f841e2aeb7b3" (UID: "9fa3c73b-db31-4204-99d3-f841e2aeb7b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:54.291495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.291406 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:54.291495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.291426 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:54.291495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.291431 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9fa3c73b-db31-4204-99d3-f841e2aeb7b3" (UID: "9fa3c73b-db31-4204-99d3-f841e2aeb7b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:34:54.291495 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.291440 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:54.292011 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.291934 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "9fa3c73b-db31-4204-99d3-f841e2aeb7b3" (UID: "9fa3c73b-db31-4204-99d3-f841e2aeb7b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:54.293275 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.293250 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "9fa3c73b-db31-4204-99d3-f841e2aeb7b3" (UID: "9fa3c73b-db31-4204-99d3-f841e2aeb7b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:34:54.293416 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.293327 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9fa3c73b-db31-4204-99d3-f841e2aeb7b3" (UID: "9fa3c73b-db31-4204-99d3-f841e2aeb7b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:34:54.293771 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.293746 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9fa3c73b-db31-4204-99d3-f841e2aeb7b3" (UID: "9fa3c73b-db31-4204-99d3-f841e2aeb7b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:34:54.293860 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.293765 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-kube-api-access-lk258" (OuterVolumeSpecName: "kube-api-access-lk258") pod "9fa3c73b-db31-4204-99d3-f841e2aeb7b3" (UID: "9fa3c73b-db31-4204-99d3-f841e2aeb7b3"). InnerVolumeSpecName "kube-api-access-lk258". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:34:54.392922 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.392887 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:54.392922 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.392918 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lk258\" (UniqueName: \"kubernetes.io/projected/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-kube-api-access-lk258\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:54.392922 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.392929 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:54.393141 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.392937 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:54.393141 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.392945 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:54.393141 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.392953 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fa3c73b-db31-4204-99d3-f841e2aeb7b3-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:34:54.509150 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.509113 2584 generic.go:356] "Generic (PLEG): container finished" podID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerID="9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e" exitCode=0 Jul 22 19:34:54.509318 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.509184 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" Jul 22 19:34:54.509318 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.509189 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" event={"ID":"9fa3c73b-db31-4204-99d3-f841e2aeb7b3","Type":"ContainerDied","Data":"9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e"} Jul 22 19:34:54.509318 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.509265 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4" event={"ID":"9fa3c73b-db31-4204-99d3-f841e2aeb7b3","Type":"ContainerDied","Data":"d7785f81c903e97a0d443ca1e0da037d16466963d82be3ff2b09dfc7c2bb2525"} Jul 22 19:34:54.509318 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.509286 2584 scope.go:117] "RemoveContainer" containerID="9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e" Jul 22 19:34:54.519612 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.519587 2584 scope.go:117] "RemoveContainer" containerID="eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509" Jul 22 19:34:54.528346 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.528325 2584 scope.go:117] "RemoveContainer" containerID="9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e" Jul 22 19:34:54.528625 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:34:54.528605 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e\": container with ID starting with 9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e not found: ID does not exist" containerID="9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e" Jul 22 19:34:54.528675 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.528635 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e"} err="failed to get container status \"9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e\": rpc error: code = NotFound desc = could not find container \"9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e\": container with ID starting with 9258c62b88cd94597cc010c5740db0b5002b5534a4efd6efe0786229b3eed52e not found: ID does not exist" Jul 22 19:34:54.528675 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.528653 2584 scope.go:117] "RemoveContainer" containerID="eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509" Jul 22 19:34:54.528888 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:34:54.528873 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509\": container with ID starting with eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509 not found: ID does not exist" containerID="eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509" Jul 22 19:34:54.528925 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.528893 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509"} err="failed to get container status \"eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509\": rpc error: code = NotFound desc = could not find container \"eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509\": container with ID starting with eda8474c0756766c8f14e056fd3fbdf71cd8d5df76ddeeba60849a77f4267509 not found: ID does not exist" Jul 22 19:34:54.538870 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.538846 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4"] Jul 22 19:34:54.544791 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.544737 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-61193d-openshift-default-75cd4885bf-x5lf4"] Jul 22 19:34:54.906989 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:54.906957 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" path="/var/lib/kubelet/pods/9fa3c73b-db31-4204-99d3-f841e2aeb7b3/volumes" Jul 22 19:34:57.721770 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.721727 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72"] Jul 22 19:34:57.722156 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.722139 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" Jul 22 19:34:57.722251 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.722179 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" Jul 22 19:34:57.722251 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.722198 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" Jul 22 19:34:57.722251 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.722205 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6763c4-a8ac-4c7d-88c7-3a3808ae7115" containerName="istio-proxy" Jul 22 19:34:57.722376 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.722282 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" Jul 22 19:34:57.726001 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.725975 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.728485 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.728464 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f3fe27-openshift-default-dockercfg-xjrhx\"" Jul 22 19:34:57.739467 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.739442 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72"] Jul 22 19:34:57.826800 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.826766 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-certs\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.827049 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.827022 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-data\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.827199 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.827080 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-envoy\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.827199 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.827161 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-socket\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.827318 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.827227 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-credential-socket\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.827318 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.827270 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25490bb1-e410-4b12-93da-32e9d614cdef-istiod-ca-cert\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.827432 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.827335 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4llxj\" (UniqueName: \"kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-kube-api-access-4llxj\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.827432 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.827362 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25490bb1-e410-4b12-93da-32e9d614cdef-istio-podinfo\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.827432 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.827392 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-istio-token\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.928648 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.928618 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-credential-socket\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.928823 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.928661 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25490bb1-e410-4b12-93da-32e9d614cdef-istiod-ca-cert\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.928823 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.928706 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4llxj\" (UniqueName: \"kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-kube-api-access-4llxj\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.928823 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.928723 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25490bb1-e410-4b12-93da-32e9d614cdef-istio-podinfo\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.928823 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.928742 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-istio-token\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.928823 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.928776 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-certs\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.929082 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.928993 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-data\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.929082 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.929067 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-envoy\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.929188 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.929084 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-credential-socket\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.929188 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.929107 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-socket\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.929188 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.929116 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-certs\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.929347 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.929334 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-data\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.929537 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.929494 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-socket\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.929666 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.929539 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25490bb1-e410-4b12-93da-32e9d614cdef-istiod-ca-cert\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.931297 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.931277 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-envoy\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.931442 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.931341 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25490bb1-e410-4b12-93da-32e9d614cdef-istio-podinfo\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.936766 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.936745 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-istio-token\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:57.936975 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:57.936953 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4llxj\" (UniqueName: \"kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-kube-api-access-4llxj\") pod \"e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:58.037741 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:58.037638 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:58.181677 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:58.181650 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72"] Jul 22 19:34:58.183491 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:34:58.183462 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25490bb1_e410_4b12_93da_32e9d614cdef.slice/crio-2a34195416850d9f73f49423ec88642fb24690aa799c3bc6538b2885ea3ab7bc WatchSource:0}: Error finding container 2a34195416850d9f73f49423ec88642fb24690aa799c3bc6538b2885ea3ab7bc: Status 404 returned error can't find the container with id 2a34195416850d9f73f49423ec88642fb24690aa799c3bc6538b2885ea3ab7bc Jul 22 19:34:58.185652 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:58.185619 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:34:58.185725 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:58.185683 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:34:58.185725 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:58.185710 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:34:58.541748 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:58.541698 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" event={"ID":"25490bb1-e410-4b12-93da-32e9d614cdef","Type":"ContainerStarted","Data":"05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6"} Jul 22 19:34:58.541938 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:58.541796 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" event={"ID":"25490bb1-e410-4b12-93da-32e9d614cdef","Type":"ContainerStarted","Data":"2a34195416850d9f73f49423ec88642fb24690aa799c3bc6538b2885ea3ab7bc"} Jul 22 19:34:58.560954 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:58.560902 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" podStartSLOduration=1.560887463 podStartE2EDuration="1.560887463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:34:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:34:58.560624777 +0000 UTC m=+2632.223199116" watchObservedRunningTime="2026-07-22 19:34:58.560887463 +0000 UTC m=+2632.223461802" Jul 22 19:34:59.038857 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:59.038819 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:59.043932 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:59.043905 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:59.547869 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:59.547837 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:34:59.549016 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:34:59.548995 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:35:07.532753 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:07.532670 2584 patch_prober.go:28] interesting pod/e2e-shared-a-488695-openshift-default-794b69b94c-dx987 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 19:35:07.532753 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:07.532735 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 19:35:08.604038 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:08.604003 2584 generic.go:356] "Generic (PLEG): container finished" podID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerID="2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1" exitCode=0 Jul 22 19:35:08.604463 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:08.604078 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" event={"ID":"8b7abac4-2551-4a85-b51e-1d65f5b04a13","Type":"ContainerDied","Data":"2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1"} Jul 22 19:35:08.604505 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:08.604497 2584 scope.go:117] "RemoveContainer" containerID="2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1" Jul 22 19:35:08.605580 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:08.605552 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:35:08.605659 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:08.605621 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:35:08.605659 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:08.605650 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:35:09.616214 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:09.616164 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" event={"ID":"8b7abac4-2551-4a85-b51e-1d65f5b04a13","Type":"ContainerStarted","Data":"37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851"} Jul 22 19:35:10.134058 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:10.134014 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:35:10.136958 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:10.134451 2584 patch_prober.go:28] interesting pod/e2e-shared-a-488695-openshift-default-794b69b94c-dx987 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 22 19:35:10.136958 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:10.136829 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 22 19:35:11.132568 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:11.132490 2584 patch_prober.go:28] interesting pod/e2e-shared-a-488695-openshift-default-794b69b94c-dx987 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 22 19:35:11.133091 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:11.132585 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 22 19:35:12.132292 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:12.132245 2584 patch_prober.go:28] interesting pod/e2e-shared-a-488695-openshift-default-794b69b94c-dx987 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 22 19:35:12.132531 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:12.132323 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 22 19:35:13.131685 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:13.131639 2584 patch_prober.go:28] interesting pod/e2e-shared-a-488695-openshift-default-794b69b94c-dx987 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 22 19:35:13.132090 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:13.131700 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 22 19:35:14.135914 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:14.135827 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:35:14.136369 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:14.136227 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:35:14.137094 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:14.137071 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:35:15.589717 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:15.589666 2584 patch_prober.go:28] interesting pod/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 19:35:15.590220 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:15.589724 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 19:35:17.667391 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:17.667357 2584 generic.go:356] "Generic (PLEG): container finished" podID="25490bb1-e410-4b12-93da-32e9d614cdef" containerID="05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6" exitCode=0 Jul 22 19:35:17.667851 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:17.667430 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" event={"ID":"25490bb1-e410-4b12-93da-32e9d614cdef","Type":"ContainerDied","Data":"05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6"} Jul 22 19:35:17.667897 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:17.667855 2584 scope.go:117] "RemoveContainer" containerID="05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6" Jul 22 19:35:17.668873 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:17.668842 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:35:17.668984 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:17.668924 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:35:17.668984 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:17.668970 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 19:35:18.674776 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:18.674733 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" event={"ID":"25490bb1-e410-4b12-93da-32e9d614cdef","Type":"ContainerStarted","Data":"40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b"} Jul 22 19:35:19.038372 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:19.038276 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:35:19.038619 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:19.038479 2584 patch_prober.go:28] interesting pod/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 22 19:35:19.038619 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:19.038565 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 22 19:35:20.038248 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:20.038207 2584 patch_prober.go:28] interesting pod/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 22 19:35:20.038715 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:20.038268 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 22 19:35:21.038793 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:21.038756 2584 patch_prober.go:28] interesting pod/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 22 19:35:21.039190 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:21.038826 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 22 19:35:22.038839 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:22.038797 2584 patch_prober.go:28] interesting pod/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 22 19:35:22.039247 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:22.038867 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 22 19:35:23.042237 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:23.042204 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:35:23.042730 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:23.042604 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:35:23.043306 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:35:23.043289 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:36:07.112542 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:36:07.112439 2584 scope.go:117] "RemoveContainer" containerID="f0be23f0a39d28331953c01f23568564939af2712e27fae83749239faf927d5e" Jul 22 19:36:09.377842 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:36:09.377816 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:36:09.385360 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:36:09.385335 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:39:07.153047 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:39:07.152966 2584 scope.go:117] "RemoveContainer" containerID="ece7d15b1a54617c3311a763b0a1bec5f8514de524325cab7f74a8cb3f420bbe" Jul 22 19:40:07.176943 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:40:07.176911 2584 scope.go:117] "RemoveContainer" containerID="9e40f85a181a391e462e4904afd07119a5b7a95528d908ebacd9f8c25b527ec9" Jul 22 19:40:07.186198 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:40:07.186175 2584 scope.go:117] "RemoveContainer" containerID="bee59c265ce974e0219547209194e95d81b56aff287bcdaa8ffe003cd658c0b9" Jul 22 19:41:09.412318 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:41:09.412292 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:41:09.420213 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:41:09.420191 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:42:00.334067 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.333989 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf"] Jul 22 19:42:00.334549 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.334328 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" Jul 22 19:42:00.334549 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.334339 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" Jul 22 19:42:00.334549 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.334428 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fa3c73b-db31-4204-99d3-f841e2aeb7b3" containerName="istio-proxy" Jul 22 19:42:00.337499 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.337481 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf" Jul 22 19:42:00.340269 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.340243 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\"" Jul 22 19:42:00.344961 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.344937 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf"] Jul 22 19:42:00.422724 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.422689 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f62f95c-2f2c-4e5f-a0d4-009a881550eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-488695-8t9lf\" (UID: \"4f62f95c-2f2c-4e5f-a0d4-009a881550eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf" Jul 22 19:42:00.523336 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.523304 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f62f95c-2f2c-4e5f-a0d4-009a881550eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-488695-8t9lf\" (UID: \"4f62f95c-2f2c-4e5f-a0d4-009a881550eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf" Jul 22 19:42:00.523959 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.523939 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f62f95c-2f2c-4e5f-a0d4-009a881550eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-488695-8t9lf\" (UID: \"4f62f95c-2f2c-4e5f-a0d4-009a881550eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf" Jul 22 19:42:00.651298 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.651221 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf" Jul 22 19:42:00.785347 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:42:00.785318 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f62f95c_2f2c_4e5f_a0d4_009a881550eb.slice/crio-511aeec12b9f0f6cfcda96bd37198a9b171e9a56e82e37e0058f0a9a1f1d2250 WatchSource:0}: Error finding container 511aeec12b9f0f6cfcda96bd37198a9b171e9a56e82e37e0058f0a9a1f1d2250: Status 404 returned error can't find the container with id 511aeec12b9f0f6cfcda96bd37198a9b171e9a56e82e37e0058f0a9a1f1d2250 Jul 22 19:42:00.785487 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.785403 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf"] Jul 22 19:42:00.828705 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:00.828679 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf" event={"ID":"4f62f95c-2f2c-4e5f-a0d4-009a881550eb","Type":"ContainerStarted","Data":"511aeec12b9f0f6cfcda96bd37198a9b171e9a56e82e37e0058f0a9a1f1d2250"} Jul 22 19:42:01.835761 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:01.835724 2584 generic.go:356] "Generic (PLEG): container finished" podID="4f62f95c-2f2c-4e5f-a0d4-009a881550eb" containerID="20d25af43193fad40934ed619095c765d979587240ca61c14dde9a34396b47f6" exitCode=0 Jul 22 19:42:01.836135 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:01.835807 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf" event={"ID":"4f62f95c-2f2c-4e5f-a0d4-009a881550eb","Type":"ContainerDied","Data":"20d25af43193fad40934ed619095c765d979587240ca61c14dde9a34396b47f6"} Jul 22 19:42:02.975571 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:02.975546 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf" Jul 22 19:42:03.046336 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:03.046300 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f62f95c-2f2c-4e5f-a0d4-009a881550eb-maas-api-service-ca\") pod \"4f62f95c-2f2c-4e5f-a0d4-009a881550eb\" (UID: \"4f62f95c-2f2c-4e5f-a0d4-009a881550eb\") " Jul 22 19:42:03.046739 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:03.046712 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f62f95c-2f2c-4e5f-a0d4-009a881550eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f62f95c-2f2c-4e5f-a0d4-009a881550eb" (UID: "4f62f95c-2f2c-4e5f-a0d4-009a881550eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:42:03.147428 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:03.147341 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f62f95c-2f2c-4e5f-a0d4-009a881550eb-maas-api-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:03.848269 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:03.848231 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf" event={"ID":"4f62f95c-2f2c-4e5f-a0d4-009a881550eb","Type":"ContainerDied","Data":"511aeec12b9f0f6cfcda96bd37198a9b171e9a56e82e37e0058f0a9a1f1d2250"} Jul 22 19:42:03.848554 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:03.848534 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="511aeec12b9f0f6cfcda96bd37198a9b171e9a56e82e37e0058f0a9a1f1d2250" Jul 22 19:42:03.848696 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:03.848261 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf" Jul 22 19:42:15.431122 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:15.431082 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf"] Jul 22 19:42:15.433895 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:15.433868 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-488695-8t9lf"] Jul 22 19:42:16.907155 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:16.907108 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f62f95c-2f2c-4e5f-a0d4-009a881550eb" path="/var/lib/kubelet/pods/4f62f95c-2f2c-4e5f-a0d4-009a881550eb/volumes" Jul 22 19:42:21.886664 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:21.886627 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987"] Jul 22 19:42:21.887078 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:21.886886 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerName="istio-proxy" containerID="cri-o://37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851" gracePeriod=30 Jul 22 19:42:22.285709 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.285627 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz"] Jul 22 19:42:22.286015 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.286002 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f62f95c-2f2c-4e5f-a0d4-009a881550eb" containerName="revoke-keys" Jul 22 19:42:22.286059 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.286017 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f62f95c-2f2c-4e5f-a0d4-009a881550eb" containerName="revoke-keys" Jul 22 19:42:22.286095 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.286073 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f62f95c-2f2c-4e5f-a0d4-009a881550eb" containerName="revoke-keys" Jul 22 19:42:22.289049 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.289032 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz" Jul 22 19:42:22.291420 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.291398 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pvvlf\"" Jul 22 19:42:22.296094 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.296069 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz"] Jul 22 19:42:22.328555 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.328506 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e35ad06-9363-4d8b-9434-78eeda016570-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz\" (UID: \"5e35ad06-9363-4d8b-9434-78eeda016570\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz" Jul 22 19:42:22.429809 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.429776 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e35ad06-9363-4d8b-9434-78eeda016570-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz\" (UID: \"5e35ad06-9363-4d8b-9434-78eeda016570\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz" Jul 22 19:42:22.430458 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.430431 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e35ad06-9363-4d8b-9434-78eeda016570-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz\" (UID: \"5e35ad06-9363-4d8b-9434-78eeda016570\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz" Jul 22 19:42:22.603440 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.603407 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz" Jul 22 19:42:22.746220 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.746190 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz"] Jul 22 19:42:22.748376 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:42:22.748347 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e35ad06_9363_4d8b_9434_78eeda016570.slice/crio-f5621d07092c2bac88c8ba1cb96595592e2c24e0bd9bf405233dbdd686f39318 WatchSource:0}: Error finding container f5621d07092c2bac88c8ba1cb96595592e2c24e0bd9bf405233dbdd686f39318: Status 404 returned error can't find the container with id f5621d07092c2bac88c8ba1cb96595592e2c24e0bd9bf405233dbdd686f39318 Jul 22 19:42:22.949362 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:22.949318 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz" event={"ID":"5e35ad06-9363-4d8b-9434-78eeda016570","Type":"ContainerStarted","Data":"f5621d07092c2bac88c8ba1cb96595592e2c24e0bd9bf405233dbdd686f39318"} Jul 22 19:42:23.956019 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:23.955985 2584 generic.go:356] "Generic (PLEG): container finished" podID="5e35ad06-9363-4d8b-9434-78eeda016570" containerID="41deed776c3e7e07448c212b9ae7c2293fd0541aee0107e45df0170b6af76780" exitCode=0 Jul 22 19:42:23.956380 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:23.956071 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz" event={"ID":"5e35ad06-9363-4d8b-9434-78eeda016570","Type":"ContainerDied","Data":"41deed776c3e7e07448c212b9ae7c2293fd0541aee0107e45df0170b6af76780"} Jul 22 19:42:25.096181 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:25.096159 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz" Jul 22 19:42:25.155510 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:25.155475 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e35ad06-9363-4d8b-9434-78eeda016570-maas-api-service-ca\") pod \"5e35ad06-9363-4d8b-9434-78eeda016570\" (UID: \"5e35ad06-9363-4d8b-9434-78eeda016570\") " Jul 22 19:42:25.155846 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:25.155823 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e35ad06-9363-4d8b-9434-78eeda016570-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e35ad06-9363-4d8b-9434-78eeda016570" (UID: "5e35ad06-9363-4d8b-9434-78eeda016570"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:42:25.257015 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:25.256929 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e35ad06-9363-4d8b-9434-78eeda016570-maas-api-service-ca\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:25.967475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:25.967436 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz" event={"ID":"5e35ad06-9363-4d8b-9434-78eeda016570","Type":"ContainerDied","Data":"f5621d07092c2bac88c8ba1cb96595592e2c24e0bd9bf405233dbdd686f39318"} Jul 22 19:42:25.967475 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:25.967460 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz" Jul 22 19:42:25.967718 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:25.967489 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5621d07092c2bac88c8ba1cb96595592e2c24e0bd9bf405233dbdd686f39318" Jul 22 19:42:27.144544 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.144502 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:42:27.178360 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178294 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-socket\") pod \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " Jul 22 19:42:27.178489 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178360 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-token\") pod \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " Jul 22 19:42:27.178489 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178385 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcf86\" (UniqueName: \"kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-kube-api-access-rcf86\") pod \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " Jul 22 19:42:27.178489 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178405 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-podinfo\") pod \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " Jul 22 19:42:27.178489 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178441 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-credential-socket\") pod \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " Jul 22 19:42:27.178796 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178490 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-certs\") pod \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " Jul 22 19:42:27.178796 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178534 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-data\") pod \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " Jul 22 19:42:27.178796 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178565 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istiod-ca-cert\") pod \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " Jul 22 19:42:27.178796 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178593 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-envoy\") pod \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\" (UID: \"8b7abac4-2551-4a85-b51e-1d65f5b04a13\") " Jul 22 19:42:27.178796 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178778 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b7abac4-2551-4a85-b51e-1d65f5b04a13" (UID: "8b7abac4-2551-4a85-b51e-1d65f5b04a13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:27.179059 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.178920 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:27.179059 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.179046 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b7abac4-2551-4a85-b51e-1d65f5b04a13" (UID: "8b7abac4-2551-4a85-b51e-1d65f5b04a13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:42:27.179330 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.179103 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b7abac4-2551-4a85-b51e-1d65f5b04a13" (UID: "8b7abac4-2551-4a85-b51e-1d65f5b04a13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:27.179330 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.179165 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b7abac4-2551-4a85-b51e-1d65f5b04a13" (UID: "8b7abac4-2551-4a85-b51e-1d65f5b04a13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:27.179877 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.179847 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b7abac4-2551-4a85-b51e-1d65f5b04a13" (UID: "8b7abac4-2551-4a85-b51e-1d65f5b04a13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:27.180777 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.180750 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b7abac4-2551-4a85-b51e-1d65f5b04a13" (UID: "8b7abac4-2551-4a85-b51e-1d65f5b04a13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:42:27.180888 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.180858 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-kube-api-access-rcf86" (OuterVolumeSpecName: "kube-api-access-rcf86") pod "8b7abac4-2551-4a85-b51e-1d65f5b04a13" (UID: "8b7abac4-2551-4a85-b51e-1d65f5b04a13"). InnerVolumeSpecName "kube-api-access-rcf86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:42:27.181144 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.181123 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b7abac4-2551-4a85-b51e-1d65f5b04a13" (UID: "8b7abac4-2551-4a85-b51e-1d65f5b04a13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:27.181386 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.181372 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b7abac4-2551-4a85-b51e-1d65f5b04a13" (UID: "8b7abac4-2551-4a85-b51e-1d65f5b04a13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:42:27.280210 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.280155 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:27.280210 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.280205 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcf86\" (UniqueName: \"kubernetes.io/projected/8b7abac4-2551-4a85-b51e-1d65f5b04a13-kube-api-access-rcf86\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:27.280210 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.280215 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:27.280210 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.280223 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:27.280210 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.280232 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:27.280570 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.280242 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:27.280570 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.280250 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:27.280570 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.280257 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b7abac4-2551-4a85-b51e-1d65f5b04a13-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:27.985956 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.985922 2584 generic.go:356] "Generic (PLEG): container finished" podID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerID="37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851" exitCode=0 Jul 22 19:42:27.986161 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.985974 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" Jul 22 19:42:27.986161 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.986011 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" event={"ID":"8b7abac4-2551-4a85-b51e-1d65f5b04a13","Type":"ContainerDied","Data":"37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851"} Jul 22 19:42:27.986161 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.986058 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987" event={"ID":"8b7abac4-2551-4a85-b51e-1d65f5b04a13","Type":"ContainerDied","Data":"5d23e5b248fd9f0cc248454e2bf8e35639479b5e431d002ff060537c04684b62"} Jul 22 19:42:27.986161 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.986101 2584 scope.go:117] "RemoveContainer" containerID="37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851" Jul 22 19:42:27.996427 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:27.996406 2584 scope.go:117] "RemoveContainer" containerID="2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1" Jul 22 19:42:28.004740 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:28.004719 2584 scope.go:117] "RemoveContainer" containerID="37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851" Jul 22 19:42:28.005050 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:42:28.005023 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851\": container with ID starting with 37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851 not found: ID does not exist" containerID="37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851" Jul 22 19:42:28.005121 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:28.005061 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851"} err="failed to get container status \"37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851\": rpc error: code = NotFound desc = could not find container \"37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851\": container with ID starting with 37ee4eab1351ea9e9e09c2e082e496fa25e888eadc894ec352310054c0626851 not found: ID does not exist" Jul 22 19:42:28.005121 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:28.005085 2584 scope.go:117] "RemoveContainer" containerID="2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1" Jul 22 19:42:28.005344 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:42:28.005324 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1\": container with ID starting with 2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1 not found: ID does not exist" containerID="2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1" Jul 22 19:42:28.005413 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:28.005348 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1"} err="failed to get container status \"2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1\": rpc error: code = NotFound desc = could not find container \"2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1\": container with ID starting with 2d4c574b96901a0225104116610714d856f44886e1924575fe7e8418a5b141a1 not found: ID does not exist" Jul 22 19:42:28.009033 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:28.009010 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987"] Jul 22 19:42:28.012692 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:28.012669 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-488695-openshift-default-794b69b94c-dx987"] Jul 22 19:42:28.906062 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:28.906029 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" path="/var/lib/kubelet/pods/8b7abac4-2551-4a85-b51e-1d65f5b04a13/volumes" Jul 22 19:42:37.357144 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:37.357106 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz"] Jul 22 19:42:37.359131 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:37.359110 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-f3fe27-dsbtz"] Jul 22 19:42:38.906443 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:38.906411 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e35ad06-9363-4d8b-9434-78eeda016570" path="/var/lib/kubelet/pods/5e35ad06-9363-4d8b-9434-78eeda016570/volumes" Jul 22 19:42:43.705386 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:43.705348 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72"] Jul 22 19:42:43.705833 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:43.705596 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" containerName="istio-proxy" containerID="cri-o://40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b" gracePeriod=30 Jul 22 19:42:48.962820 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:48.962791 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:42:49.079876 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.079844 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-certs\") pod \"25490bb1-e410-4b12-93da-32e9d614cdef\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " Jul 22 19:42:49.080063 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.079886 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-credential-socket\") pod \"25490bb1-e410-4b12-93da-32e9d614cdef\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " Jul 22 19:42:49.080063 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.079907 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-envoy\") pod \"25490bb1-e410-4b12-93da-32e9d614cdef\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " Jul 22 19:42:49.080063 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.079932 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25490bb1-e410-4b12-93da-32e9d614cdef-istiod-ca-cert\") pod \"25490bb1-e410-4b12-93da-32e9d614cdef\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " Jul 22 19:42:49.080063 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.079978 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4llxj\" (UniqueName: \"kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-kube-api-access-4llxj\") pod \"25490bb1-e410-4b12-93da-32e9d614cdef\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " Jul 22 19:42:49.080063 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080020 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25490bb1-e410-4b12-93da-32e9d614cdef-istio-podinfo\") pod \"25490bb1-e410-4b12-93da-32e9d614cdef\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " Jul 22 19:42:49.080341 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080113 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-data\") pod \"25490bb1-e410-4b12-93da-32e9d614cdef\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " Jul 22 19:42:49.080341 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080154 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-istio-token\") pod \"25490bb1-e410-4b12-93da-32e9d614cdef\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " Jul 22 19:42:49.080341 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080191 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-socket\") pod \"25490bb1-e410-4b12-93da-32e9d614cdef\" (UID: \"25490bb1-e410-4b12-93da-32e9d614cdef\") " Jul 22 19:42:49.080341 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080195 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "25490bb1-e410-4b12-93da-32e9d614cdef" (UID: "25490bb1-e410-4b12-93da-32e9d614cdef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:49.080341 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080220 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "25490bb1-e410-4b12-93da-32e9d614cdef" (UID: "25490bb1-e410-4b12-93da-32e9d614cdef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:49.080632 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080391 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25490bb1-e410-4b12-93da-32e9d614cdef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "25490bb1-e410-4b12-93da-32e9d614cdef" (UID: "25490bb1-e410-4b12-93da-32e9d614cdef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:42:49.080632 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080497 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "25490bb1-e410-4b12-93da-32e9d614cdef" (UID: "25490bb1-e410-4b12-93da-32e9d614cdef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:49.080632 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080509 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-certs\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:49.080632 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080559 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-credential-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:49.080632 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080572 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25490bb1-e410-4b12-93da-32e9d614cdef-istiod-ca-cert\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:49.081059 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.080993 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-data" (OuterVolumeSpecName: "istio-data") pod "25490bb1-e410-4b12-93da-32e9d614cdef" (UID: "25490bb1-e410-4b12-93da-32e9d614cdef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:49.082464 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.082440 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "25490bb1-e410-4b12-93da-32e9d614cdef" (UID: "25490bb1-e410-4b12-93da-32e9d614cdef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:42:49.082464 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.082441 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25490bb1-e410-4b12-93da-32e9d614cdef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "25490bb1-e410-4b12-93da-32e9d614cdef" (UID: "25490bb1-e410-4b12-93da-32e9d614cdef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:42:49.082861 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.082831 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-istio-token" (OuterVolumeSpecName: "istio-token") pod "25490bb1-e410-4b12-93da-32e9d614cdef" (UID: "25490bb1-e410-4b12-93da-32e9d614cdef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:42:49.082861 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.082831 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-kube-api-access-4llxj" (OuterVolumeSpecName: "kube-api-access-4llxj") pod "25490bb1-e410-4b12-93da-32e9d614cdef" (UID: "25490bb1-e410-4b12-93da-32e9d614cdef"). InnerVolumeSpecName "kube-api-access-4llxj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:42:49.095879 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.095854 2584 generic.go:356] "Generic (PLEG): container finished" podID="25490bb1-e410-4b12-93da-32e9d614cdef" containerID="40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b" exitCode=0 Jul 22 19:42:49.096000 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.095899 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" Jul 22 19:42:49.096000 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.095900 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" event={"ID":"25490bb1-e410-4b12-93da-32e9d614cdef","Type":"ContainerDied","Data":"40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b"} Jul 22 19:42:49.096074 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.096024 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72" event={"ID":"25490bb1-e410-4b12-93da-32e9d614cdef","Type":"ContainerDied","Data":"2a34195416850d9f73f49423ec88642fb24690aa799c3bc6538b2885ea3ab7bc"} Jul 22 19:42:49.096074 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.096051 2584 scope.go:117] "RemoveContainer" containerID="40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b" Jul 22 19:42:49.105284 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.105268 2584 scope.go:117] "RemoveContainer" containerID="05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6" Jul 22 19:42:49.113603 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.113580 2584 scope.go:117] "RemoveContainer" containerID="40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b" Jul 22 19:42:49.113881 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:42:49.113861 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b\": container with ID starting with 40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b not found: ID does not exist" containerID="40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b" Jul 22 19:42:49.113957 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.113893 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b"} err="failed to get container status \"40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b\": rpc error: code = NotFound desc = could not find container \"40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b\": container with ID starting with 40e18b0c54022ffa095fb2f4b294936419c10d4b864030adc9c92b7c6679cc0b not found: ID does not exist" Jul 22 19:42:49.113957 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.113918 2584 scope.go:117] "RemoveContainer" containerID="05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6" Jul 22 19:42:49.114145 ip-10-0-142-177 kubenswrapper[2584]: E0722 19:42:49.114128 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6\": container with ID starting with 05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6 not found: ID does not exist" containerID="05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6" Jul 22 19:42:49.114188 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.114152 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6"} err="failed to get container status \"05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6\": rpc error: code = NotFound desc = could not find container \"05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6\": container with ID starting with 05a08afcbfdf7dd4e634fb076a24edbc4be593e8327113a4ebafb27c4ea407e6 not found: ID does not exist" Jul 22 19:42:49.118476 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.118453 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72"] Jul 22 19:42:49.123737 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.123717 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f3fe27-openshift-default-855947cf9-xqn72"] Jul 22 19:42:49.181309 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.181275 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-istio-token\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:49.181309 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.181305 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-workload-socket\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:49.181309 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.181315 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-envoy\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:49.181550 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.181324 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4llxj\" (UniqueName: \"kubernetes.io/projected/25490bb1-e410-4b12-93da-32e9d614cdef-kube-api-access-4llxj\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:49.181550 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.181332 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25490bb1-e410-4b12-93da-32e9d614cdef-istio-podinfo\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:49.181550 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:49.181340 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25490bb1-e410-4b12-93da-32e9d614cdef-istio-data\") on node \"ip-10-0-142-177.ec2.internal\" DevicePath \"\"" Jul 22 19:42:50.211677 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:50.211645 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8c7c85cd9-6plrk_e940eddf-66f1-4ade-ac0a-f973e3b0c906/manager/0.log" Jul 22 19:42:50.861389 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:50.861357 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64cd98d4d8-nt5d8_9c181f18-bdc9-4bb2-a871-37259a0ad2f9/copy-manifests/0.log" Jul 22 19:42:50.868166 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:50.868141 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64cd98d4d8-nt5d8_9c181f18-bdc9-4bb2-a871-37259a0ad2f9/manager/0.log" Jul 22 19:42:50.906755 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:50.906720 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" path="/var/lib/kubelet/pods/25490bb1-e410-4b12-93da-32e9d614cdef/volumes" Jul 22 19:42:51.239878 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:51.239800 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-569cdb7f56-xr224_e54fe280-64a1-4210-ba2d-4df8469a74d5/maas-api/0.log" Jul 22 19:42:52.411499 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.411467 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf_410de48d-1fe6-4dde-bdd1-35d9d05cd0fa/util/0.log" Jul 22 19:42:52.418695 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.418658 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf_410de48d-1fe6-4dde-bdd1-35d9d05cd0fa/pull/0.log" Jul 22 19:42:52.424202 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.424183 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf_410de48d-1fe6-4dde-bdd1-35d9d05cd0fa/extract/0.log" Jul 22 19:42:52.546976 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.546935 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv_2dca680d-25d4-41e9-b2bd-7aa4cf63b0af/extract/0.log" Jul 22 19:42:52.552635 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.552611 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv_2dca680d-25d4-41e9-b2bd-7aa4cf63b0af/util/0.log" Jul 22 19:42:52.558701 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.558668 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv_2dca680d-25d4-41e9-b2bd-7aa4cf63b0af/pull/0.log" Jul 22 19:42:52.675884 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.675811 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt_dcae49ee-0270-44ff-bf97-67ef5d318210/util/0.log" Jul 22 19:42:52.682015 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.681992 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt_dcae49ee-0270-44ff-bf97-67ef5d318210/pull/0.log" Jul 22 19:42:52.687776 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.687751 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt_dcae49ee-0270-44ff-bf97-67ef5d318210/extract/0.log" Jul 22 19:42:52.803642 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.803612 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw_f94d5404-cebf-4372-b51c-f0be34e2df6a/util/0.log" Jul 22 19:42:52.809654 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.809629 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw_f94d5404-cebf-4372-b51c-f0be34e2df6a/pull/0.log" Jul 22 19:42:52.815478 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:52.815456 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw_f94d5404-cebf-4372-b51c-f0be34e2df6a/extract/0.log" Jul 22 19:42:54.134353 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:54.134321 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4fp5fh_a1fe132e-14d8-4ff6-bf56-c0285356b9f6/istio-proxy/0.log" Jul 22 19:42:54.264373 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:54.264342 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-n57v4_8637487b-71d7-44e0-a324-ade94750d656/discovery/0.log" Jul 22 19:42:54.611823 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:54.611778 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7hwcv_3179ecc9-7c67-46ac-bb01-09336685a9a3/istio-proxy/1.log" Jul 22 19:42:55.558758 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:42:55.558729 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wbqtd_d132cf56-cbc5-4d55-b04f-4e90a7b0194e/main/0.log" Jul 22 19:43:04.448819 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:04.448790 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-247b8_5bf105fc-1d98-4507-9129-3a5d1027d705/global-pull-secret-syncer/0.log" Jul 22 19:43:04.673809 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:04.673775 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vmpdl_04d29c18-d8ce-4440-8438-3e217fb921df/konnectivity-agent/0.log" Jul 22 19:43:04.750753 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:04.750659 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-177.ec2.internal_2710698bde29e793f9633bc0d191b7e7/haproxy/0.log" Jul 22 19:43:09.188727 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.188690 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf_410de48d-1fe6-4dde-bdd1-35d9d05cd0fa/extract/0.log" Jul 22 19:43:09.211720 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.211692 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf_410de48d-1fe6-4dde-bdd1-35d9d05cd0fa/util/0.log" Jul 22 19:43:09.232619 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.232591 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bt7nf_410de48d-1fe6-4dde-bdd1-35d9d05cd0fa/pull/0.log" Jul 22 19:43:09.259420 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.259386 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv_2dca680d-25d4-41e9-b2bd-7aa4cf63b0af/extract/0.log" Jul 22 19:43:09.281060 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.281032 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv_2dca680d-25d4-41e9-b2bd-7aa4cf63b0af/util/0.log" Jul 22 19:43:09.311101 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.311080 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6znv_2dca680d-25d4-41e9-b2bd-7aa4cf63b0af/pull/0.log" Jul 22 19:43:09.342935 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.342909 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt_dcae49ee-0270-44ff-bf97-67ef5d318210/extract/0.log" Jul 22 19:43:09.366607 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.366578 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt_dcae49ee-0270-44ff-bf97-67ef5d318210/util/0.log" Jul 22 19:43:09.388496 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.388469 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737xfmt_dcae49ee-0270-44ff-bf97-67ef5d318210/pull/0.log" Jul 22 19:43:09.414444 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.414422 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw_f94d5404-cebf-4372-b51c-f0be34e2df6a/extract/0.log" Jul 22 19:43:09.452561 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.452461 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw_f94d5404-cebf-4372-b51c-f0be34e2df6a/util/0.log" Jul 22 19:43:09.475332 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:09.475305 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pgrgw_f94d5404-cebf-4372-b51c-f0be34e2df6a/pull/0.log" Jul 22 19:43:12.334199 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:12.334171 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-mzswh_bc0db99a-e7b2-477a-adf5-2136fa37ac45/cluster-monitoring-operator/0.log" Jul 22 19:43:12.366272 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:12.366251 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-6lcng_4b6a3e3f-cc74-414a-9c3f-964f79b650d3/kube-state-metrics/0.log" Jul 22 19:43:12.387002 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:12.386982 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-6lcng_4b6a3e3f-cc74-414a-9c3f-964f79b650d3/kube-rbac-proxy-main/0.log" Jul 22 19:43:12.407223 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:12.407201 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-6lcng_4b6a3e3f-cc74-414a-9c3f-964f79b650d3/kube-rbac-proxy-self/0.log" Jul 22 19:43:12.445931 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:12.445910 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8bd746b97-mxgnr_70614419-09b5-4fea-9d3b-aa9bccf3c6c5/metrics-server/0.log" Jul 22 19:43:12.505208 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:12.505188 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d96jc_df232a8d-6dd2-495e-857a-15a38b6d84a8/node-exporter/0.log" Jul 22 19:43:12.529335 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:12.529320 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d96jc_df232a8d-6dd2-495e-857a-15a38b6d84a8/kube-rbac-proxy/0.log" Jul 22 19:43:12.552735 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:12.552714 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d96jc_df232a8d-6dd2-495e-857a-15a38b6d84a8/init-textfile/0.log" Jul 22 19:43:14.384060 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384020 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs"] Jul 22 19:43:14.384502 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384431 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" containerName="istio-proxy" Jul 22 19:43:14.384502 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384451 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" containerName="istio-proxy" Jul 22 19:43:14.384502 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384469 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerName="istio-proxy" Jul 22 19:43:14.384502 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384478 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerName="istio-proxy" Jul 22 19:43:14.384502 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384501 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e35ad06-9363-4d8b-9434-78eeda016570" containerName="revoke-keys" Jul 22 19:43:14.384802 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384510 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e35ad06-9363-4d8b-9434-78eeda016570" containerName="revoke-keys" Jul 22 19:43:14.384802 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384648 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" containerName="istio-proxy" Jul 22 19:43:14.384802 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384660 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerName="istio-proxy" Jul 22 19:43:14.384802 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384666 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="25490bb1-e410-4b12-93da-32e9d614cdef" containerName="istio-proxy" Jul 22 19:43:14.384802 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384673 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b7abac4-2551-4a85-b51e-1d65f5b04a13" containerName="istio-proxy" Jul 22 19:43:14.384802 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.384679 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e35ad06-9363-4d8b-9434-78eeda016570" containerName="revoke-keys" Jul 22 19:43:14.389255 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.389236 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.391828 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.391780 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zws4d\"/\"default-dockercfg-z92fq\"" Jul 22 19:43:14.392786 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.392767 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zws4d\"/\"kube-root-ca.crt\"" Jul 22 19:43:14.392893 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.392875 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zws4d\"/\"openshift-service-ca.crt\"" Jul 22 19:43:14.397147 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.396761 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs"] Jul 22 19:43:14.501003 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.500968 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-podres\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.501136 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.501076 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-lib-modules\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.501136 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.501108 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-proc\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.501222 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.501160 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-sys\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.501222 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.501175 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvfmz\" (UniqueName: \"kubernetes.io/projected/190ea6eb-692a-405c-82b3-70310df5dedf-kube-api-access-nvfmz\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.602221 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.602195 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-sys\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.602332 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.602243 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvfmz\" (UniqueName: \"kubernetes.io/projected/190ea6eb-692a-405c-82b3-70310df5dedf-kube-api-access-nvfmz\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.602332 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.602277 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-podres\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.602332 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.602312 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-sys\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.602332 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.602330 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-lib-modules\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.602470 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.602406 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-podres\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.602470 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.602435 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-lib-modules\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.602576 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.602497 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-proc\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.602630 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.602608 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/190ea6eb-692a-405c-82b3-70310df5dedf-proc\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.610602 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.610579 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvfmz\" (UniqueName: \"kubernetes.io/projected/190ea6eb-692a-405c-82b3-70310df5dedf-kube-api-access-nvfmz\") pod \"perf-node-gather-daemonset-vgqjs\" (UID: \"190ea6eb-692a-405c-82b3-70310df5dedf\") " pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.700821 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.700762 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:14.835490 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:14.835460 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs"] Jul 22 19:43:14.837176 ip-10-0-142-177 kubenswrapper[2584]: W0722 19:43:14.837146 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod190ea6eb_692a_405c_82b3_70310df5dedf.slice/crio-d9dba3f566ed1b149154039ead1ac93b9c3c6722ac2022d3a1b2f12156b30c2b WatchSource:0}: Error finding container d9dba3f566ed1b149154039ead1ac93b9c3c6722ac2022d3a1b2f12156b30c2b: Status 404 returned error can't find the container with id d9dba3f566ed1b149154039ead1ac93b9c3c6722ac2022d3a1b2f12156b30c2b Jul 22 19:43:15.093338 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:15.093303 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/2.log" Jul 22 19:43:15.101310 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:15.101256 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-lqrlz_7547bde7-d558-4a05-bdad-e06e98fdcb2a/console-operator/3.log" Jul 22 19:43:15.239371 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:15.239340 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" event={"ID":"190ea6eb-692a-405c-82b3-70310df5dedf","Type":"ContainerStarted","Data":"7f9946e5422771bc489927cc68711914689559e380b86942c4922b8dfad423fc"} Jul 22 19:43:15.239509 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:15.239395 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:15.239509 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:15.239414 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" event={"ID":"190ea6eb-692a-405c-82b3-70310df5dedf","Type":"ContainerStarted","Data":"d9dba3f566ed1b149154039ead1ac93b9c3c6722ac2022d3a1b2f12156b30c2b"} Jul 22 19:43:15.255627 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:15.255583 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" podStartSLOduration=1.2555710869999999 podStartE2EDuration="1.255571087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 19:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:43:15.253938799 +0000 UTC m=+3128.916513139" watchObservedRunningTime="2026-07-22 19:43:15.255571087 +0000 UTC m=+3128.918145426" Jul 22 19:43:15.612121 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:15.612083 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78448657d6-wqtq6_495e2b8c-43c5-49e4-b170-f7ce121d26cd/console/0.log" Jul 22 19:43:15.650869 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:15.650838 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-tjm7d_f564341b-cc25-48df-92cd-145b2b762c76/download-server/0.log" Jul 22 19:43:17.369034 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:17.368999 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7lk8l_b6144392-e95f-4cf4-973c-f73ce61a0780/dns/0.log" Jul 22 19:43:17.389304 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:17.389282 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7lk8l_b6144392-e95f-4cf4-973c-f73ce61a0780/kube-rbac-proxy/0.log" Jul 22 19:43:17.555406 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:17.555376 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lpxbp_62f16011-c0a3-42ac-ace9-fa028fc57f76/dns-node-resolver/0.log" Jul 22 19:43:18.201498 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:18.201449 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g2xmq_5c280d2a-6833-4775-a794-317b88a86303/node-ca/0.log" Jul 22 19:43:19.101685 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:19.101648 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4fp5fh_a1fe132e-14d8-4ff6-bf56-c0285356b9f6/istio-proxy/0.log" Jul 22 19:43:19.329547 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:19.329492 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-n57v4_8637487b-71d7-44e0-a324-ade94750d656/discovery/0.log" Jul 22 19:43:19.404153 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:19.404039 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7hwcv_3179ecc9-7c67-46ac-bb01-09336685a9a3/istio-proxy/0.log" Jul 22 19:43:19.457967 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:19.457941 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7hwcv_3179ecc9-7c67-46ac-bb01-09336685a9a3/istio-proxy/1.log" Jul 22 19:43:20.236858 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:20.236823 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7qjbt_5037332a-7cf0-489d-b14c-ce356a5a3c4d/serve-healthcheck-canary/0.log" Jul 22 19:43:20.917721 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:20.917680 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kpw96_9bd8f044-f8df-42a4-abf9-59035972b0f4/kube-rbac-proxy/0.log" Jul 22 19:43:20.938635 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:20.938610 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kpw96_9bd8f044-f8df-42a4-abf9-59035972b0f4/exporter/0.log" Jul 22 19:43:20.963425 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:20.963402 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kpw96_9bd8f044-f8df-42a4-abf9-59035972b0f4/extractor/0.log" Jul 22 19:43:21.255232 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:21.255151 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zws4d/perf-node-gather-daemonset-vgqjs" Jul 22 19:43:23.836595 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:23.836564 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8c7c85cd9-6plrk_e940eddf-66f1-4ade-ac0a-f973e3b0c906/manager/0.log" Jul 22 19:43:24.202706 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:24.202675 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64cd98d4d8-nt5d8_9c181f18-bdc9-4bb2-a871-37259a0ad2f9/manager/0.log" Jul 22 19:43:24.222180 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:24.222160 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64cd98d4d8-nt5d8_9c181f18-bdc9-4bb2-a871-37259a0ad2f9/copy-manifests/0.log" Jul 22 19:43:25.196355 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:25.196324 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4fp5fh_a1fe132e-14d8-4ff6-bf56-c0285356b9f6/istio-proxy/0.log" Jul 22 19:43:25.421142 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:25.421113 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-n57v4_8637487b-71d7-44e0-a324-ade94750d656/discovery/0.log" Jul 22 19:43:25.435876 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:25.435848 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c9874fb98-gt9tk_8c38839a-dab6-4cbe-b004-641a60a1138a/manager/0.log" Jul 22 19:43:25.463769 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:25.463697 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7hwcv_3179ecc9-7c67-46ac-bb01-09336685a9a3/istio-proxy/0.log" Jul 22 19:43:25.487601 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:25.487565 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-pcp6l_9ef95fea-e92b-4dd4-9d1f-0b0218a3494d/openshift-lws-operator/0.log" Jul 22 19:43:25.519236 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:25.519214 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7hwcv_3179ecc9-7c67-46ac-bb01-09336685a9a3/istio-proxy/1.log" Jul 22 19:43:26.812317 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:26.812228 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4fp5fh_a1fe132e-14d8-4ff6-bf56-c0285356b9f6/istio-proxy/0.log" Jul 22 19:43:27.035212 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:27.035175 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-n57v4_8637487b-71d7-44e0-a324-ade94750d656/discovery/0.log" Jul 22 19:43:27.087073 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:27.087045 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7hwcv_3179ecc9-7c67-46ac-bb01-09336685a9a3/istio-proxy/0.log" Jul 22 19:43:27.141188 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:27.141154 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7hwcv_3179ecc9-7c67-46ac-bb01-09336685a9a3/istio-proxy/1.log" Jul 22 19:43:31.757553 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:31.757507 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-867pp_f4873f3b-f674-41ca-bc4a-44595135d951/kube-multus/0.log" Jul 22 19:43:31.813952 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:31.813927 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bh8z5_124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac/kube-multus-additional-cni-plugins/0.log" Jul 22 19:43:31.833694 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:31.833672 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bh8z5_124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac/egress-router-binary-copy/0.log" Jul 22 19:43:31.853590 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:31.853567 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bh8z5_124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac/cni-plugins/0.log" Jul 22 19:43:31.873471 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:31.873449 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bh8z5_124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac/bond-cni-plugin/0.log" Jul 22 19:43:31.893599 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:31.893582 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bh8z5_124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac/routeoverride-cni/0.log" Jul 22 19:43:31.913892 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:31.913876 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bh8z5_124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac/whereabouts-cni-bincopy/0.log" Jul 22 19:43:31.933333 ip-10-0-142-177 kubenswrapper[2584]: I0722 19:43:31.933315 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bh8z5_124b4c34-5cd8-4cc9-9be9-ff1f40cdb3ac/whereabouts-cni/0.log"